> On Apr 20, 2017, at 10:36 PM, knarra <kna...@redhat.com> wrote:
>> The installer claimed it did, but I believe it didn’t. Below the error from
>> my original email, there’s the below (apologies for not including it
>> earlier; I missed it). Note: 04ff4cf1-135a-4918-9a1f-8023322f89a3 is the HE
>> - I’m pretty sure it is complaining about itself. (In any case, I verified
>> that there are no other VMs running with both virsh and vdsClient.)
^^^
>> 2017-04-19 12:27:02 DEBUG otopi.context context._executeMethod:128 Stage
>> late_setup METHOD otopi.plugins.gr_he_setup.vm.runvm.Plugin._late_setup
>> 2017-04-19 12:27:02 DEBUG otopi.plugins.gr_he_setup.vm.runvm
>> runvm._late_setup:83 {'status': {'message': 'Done', 'code': 0}, 'items':
>> [u'04ff4cf1-135a-4918-9a1f-8023322f89a3']}
>> 2017-04-19 12:27:02 ERROR otopi.plugins.gr_he_setup.vm.runvm
>> runvm._late_setup:91 The following VMs have been found:
>> 04ff4cf1-135a-4918-9a1f-8023322f89a3
>> 2017-04-19 12:27:02 DEBUG otopi.context context._executeMethod:142 method
>> exception
>> Traceback (most recent call last):
>> File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in
>> _executeMethod
>> method['method']()
>> File
>> "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-setup/vm/runvm.py",
>> line 95, in _late_setup
>> _('Cannot setup Hosted Engine with other VMs running')
>> RuntimeError: Cannot setup Hosted Engine with other VMs running
>> 2017-04-19 12:27:02 ERROR otopi.context context._executeMethod:151 Failed to
>> execute stage 'Environment setup': Cannot setup Hosted Engine with other VMs
>> running
>> 2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:760
>> ENVIRONMENT DUMP - BEGIN
>> 2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:770 ENV
>> BASE/error=bool:'True'
>> 2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:770 ENV
>> BASE/exceptionInfo=list:'[(<type 'exceptions.RuntimeError'>,
>> RuntimeError('Cannot setup Hosted Engine with other VMs running',),
>> <traceback object at 0x4358098>)]'
>> 2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:774
>> ENVIRONMENT DUMP - END
> James, generally this issue happens when the setup failed once and you tried
> re running it again. Can you clean it and deploy it again? HE should come
> up successfully. Below are the steps for cleaning it up.
Knarra,
I realize that. However, that is not the situation in my case. See above, at
the mark - the UUID it is complaining about is the UUID of the hosted-engine it
just installed. From the answers file generated from the run (whole thing
below):
>>>> OVEHOSTED_VM/vmUUID=str:04ff4cf1-135a-4918-9a1f-8023322f89a3
Also see the WARNs I mentioned previously, quoted below. Excerpt:
>>>> Apr 19 12:29:20 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root WARN
>>>> File:
>>>> /var/lib/libvirt/qemu/channels/04ff4cf1-135a-4918-9a1f-8023322f89a3.com.redhat.rhevm.vdsm
>>>> already removed
>>>> Apr 19 12:29:20 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root WARN
>>>> File:
>>>> /var/lib/libvirt/qemu/channels/04ff4cf1-135a-4918-9a1f-8023322f89a3.org.qemu.guest_agent.0
>>>> already removed
>>>> Apr 19 12:29:30 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect
>>>> to broker, the number of errors has exceeded the limit (1)
I’m not clear on what it is attempting to do there, but it seems relevant.
I know there is no failed install left on the gluster volume, because when I
attempt an install, part of my scripted prep process is deleting and recreating
the Gluster volume. The below instructions are more or less what I’m doing
already in a script[1]. (the gluster portion of the script process is: stop the
volume, delete the volume, remove the mount point directory to avoid Gluster’s
xattr problem with recycling directories, recreate the directory, change perms,
create the volume, start the volume, set Ovirt-recc’ed volume options.)
-j
[1] We have a requirement for automated setup of all production resources, so
all of this ends up being scripted.
> 1) vdsClient -s 0 list table | awk '{print $1}' | xargs vdsClient -s 0 destroy
>
> 2) stop the volume and delete all the information inside the bricks from all
> the hosts
>
> 3) try to umount storage from /rhev/data-center/mnt/ - umount -f
> /rhev/data-center/mnt/ if it is mounted
>
> 4) remove all dirs from /rhev/data-center/mnt/ - rm -rf
> /rhev/data-center/mnt/*
>
> 5) start volume again and start the deployment.
>
> Thanks
> kasturi
>>
>>
>>>> If I start it manually, the default DC is down, the default cluster has
>>>> the installation host in the cluster, there is no storage, and the VM
>>>> doesn’t show up in the GUI. In this install run, I have not yet started
>>>> the engine manually.
>>> you wont be seeing HE vm until HE storage is imported into the UI. HE
>>> storage will be automatically imported into the UI (which will import HE vm
>>> too )once a master domain is present .
>> Sure; I’m just attempting to provide context.
>>
>>>> I assume this is related to the errors in ovirt-hosted-engine-setup.log,
>>>> below. (The timestamps are confusing; it looks like the Python errors are
>>>> logged some time after they’re captured or something.) The HA broker and
>>>> agent logs just show them looping in the sequence below.
>>>>
>>>> Is there a decent way to pick this up and continue? If not, how do I make
>>>> this work?
>>> Can you please check the following things.
>>>
>>> 1) is glusterd running on all the nodes ? 'systemctl status glistered’
>>> 2) Are you able to connect to your storage server which is ovirt_engine in
>>> your case.
>>> 3) Can you check if all the brick process in the volume is up ?
>>
>> 1) Verified that glusterd is running on all three nodes.
>>
>> 2)
>> [root@sc5-thing-1]# mount -tglusterfs sc5-gluster-1:/ovirt_engine
>> /mnt/ovirt_engine
>> [root@sc5-thing-1]# df -h
>> Filesystem Size Used Avail Use% Mounted on
>> […]
>> sc5-gluster-1:/ovirt_engine 300G 2.6G 298G 1% /mnt/ovirt_engine
>>
>>
>> 3)
>> [root@sc5-gluster-1 jlawrence]# gluster volume status
>> Status of volume: ovirt_engine
>> Gluster process TCP Port RDMA Port Online Pid
>> ------------------------------------------------------------------------------
>> Brick sc5-gluster-1:/gluster-bricks/ovirt_e
>> ngine/ovirt_engine-1 49217 0 Y
>> 22102
>> Brick sc5-gluster-2:/gluster-bricks/ovirt_e
>> ngine/ovirt_engine-1 49157 0 Y
>> 37842
>> Brick sc5-gluster-3:/gluster-bricks/ovirt_e
>> ngine/ovirt_engine-1 49157 0 Y
>> 112018
>> Self-heal Daemon on localhost N/A N/A Y
>> 113364
>> Self-heal Daemon on sc5-gluster-2.squaretra
>> de.com N/A N/A Y
>> 19344
>> Self-heal Daemon on sc5-gluster-3.squaretra
>> de.com N/A N/A Y
>> 37956
>>
>> Task Status of Volume ovirt_engine
>> ------------------------------------------------------------------------------
>> There are no active volume tasks
>>
>> Volume ovirt_images is not started
>>
>>
>>
>>
>>> Thanks
>>> kasturi.
>> Thank you!
>>
>> -j
>>
>>
>>>> Thanks,
>>>>
>>>> -j
>>>>
>>>> - - - - ovirt-hosted-engine-setup.log snippet: - - - -
>>>>
>>>> 2017-04-19 12:29:55 DEBUG otopi.context context._executeMethod:128 Stage
>>>> late_setup METHOD
>>>> otopi.plugins.gr_he_setup.system.vdsmenv.Plugin._late_setup
>>>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>>>> systemd.status:90 check service vdsmd status
>>>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>>>> plugin.executeRaw:813 execute: ('/bin/systemctl', 'status',
>>>> 'vdsmd.service'), executable='None', cwd='None', env=None
>>>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>>>> plugin.executeRaw:863 execute-result: ('/bin/systemctl', 'status',
>>>> 'vdsmd.service'), rc=0
>>>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>>>> plugin.execute:921 execute-output: ('/bin/systemctl', 'status',
>>>> 'vdsmd.service') stdout:
>>>> ● vdsmd.service - Virtual Desktop Server Manager
>>>> Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor
>>>> preset: enabled)
>>>> Active: active (running) since Wed 2017-04-19 12:26:59 PDT; 2min 55s ago
>>>> Process: 67370 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh
>>>> --post-stop (code=exited, status=0/SUCCESS)
>>>> Process: 69995 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh
>>>> --pre-start (code=exited, status=0/SUCCESS)
>>>> Main PID: 70062 (vdsm)
>>>> CGroup: /system.slice/vdsmd.service
>>>> └─70062 /usr/bin/python2 /usr/share/vdsm/vdsm
>>>>
>>>> Apr 19 12:29:00 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect
>>>> to broker, the number of errors has exceeded the limit (1)
>>>> Apr 19 12:29:00 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>>>> failed to retrieve Hosted Engine HA info
>>>> Traceback (most
>>>> recent call last):
>>>> File
>>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in
>>>> _getHaInfo
>>>> stats =
>>>> instance.get_all_stats()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>>>> line 102, in get_all_stats
>>>> with
>>>> broker.connection(self._retries, self._wait):
>>>> File
>>>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>>>> return
>>>> self.gen.next()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 99, in connection
>>>>
>>>> self.connect(retries, wait)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 78, in connect
>>>> raise
>>>> BrokerConnectionError(error_msg)
>>>>
>>>> BrokerConnectionError: Failed to connect to broker, the number of errors
>>>> has exceeded the limit (1)
>>>> Apr 19 12:29:15 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect
>>>> to broker, the number of errors has exceeded the limit (1)
>>>> Apr 19 12:29:15 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>>>> failed to retrieve Hosted Engine HA info
>>>> Traceback (most
>>>> recent call last):
>>>> File
>>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in
>>>> _getHaInfo
>>>> stats =
>>>> instance.get_all_stats()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>>>> line 102, in get_all_stats
>>>> with
>>>> broker.connection(self._retries, self._wait):
>>>> File
>>>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>>>> return
>>>> self.gen.next()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 99, in connection
>>>>
>>>> self.connect(retries, wait)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 78, in connect
>>>> raise
>>>> BrokerConnectionError(error_msg)
>>>>
>>>> BrokerConnectionError: Failed to connect to broker, the number of errors
>>>> has exceeded the limit (1)
>>>> Apr 19 12:29:20 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root WARN
>>>> File:
>>>> /var/lib/libvirt/qemu/channels/04ff4cf1-135a-4918-9a1f-8023322f89a3.com.redhat.rhevm.vdsm
>>>> already removed
>>>> Apr 19 12:29:20 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root WARN
>>>> File:
>>>> /var/lib/libvirt/qemu/channels/04ff4cf1-135a-4918-9a1f-8023322f89a3.org.qemu.guest_agent.0
>>>> already removed
>>>> Apr 19 12:29:30 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect
>>>> to broker, the number of errors has exceeded the limit (1)
>>>> Apr 19 12:29:30 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>>>> failed to retrieve Hosted Engine HA info
>>>> Traceback (most
>>>> recent call last):
>>>> File
>>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in
>>>> _getHaInfo
>>>> stats =
>>>> instance.get_all_stats()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>>>> line 102, in get_all_stats
>>>> with
>>>> broker.connection(self._retries, self._wait):
>>>> File
>>>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>>>> return
>>>> self.gen.next()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 99, in connection
>>>>
>>>> self.connect(retries, wait)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 78, in connect
>>>> raise
>>>> BrokerConnectionError(error_msg)
>>>>
>>>> BrokerConnectionError: Failed to connect to broker, the number of errors
>>>> has exceeded the limit (1)
>>>> Apr 19 12:29:45 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>>>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect
>>>> to broker, the number of errors has exceeded the limit (1)
>>>> Apr 19 12:29:45 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>>>> failed to retrieve Hosted Engine HA info
>>>> Traceback (most
>>>> recent call last):
>>>> File
>>>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in
>>>> _getHaInfo
>>>> stats =
>>>> instance.get_all_stats()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>>>> line 102, in get_all_stats
>>>> with
>>>> broker.connection(self._retries, self._wait):
>>>> File
>>>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>>>> return
>>>> self.gen.next()
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 99, in connection
>>>>
>>>> self.connect(retries, wait)
>>>> File
>>>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>>>> line 78, in connect
>>>> raise
>>>> BrokerConnectionError(error_msg)
>>>>
>>>> BrokerConnectionError: Failed to connect to broker, the number of errors
>>>> has exceeded the limit (1)
>>>>
>>>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>>>> plugin.execute:926 execute-output: ('/bin/systemctl', 'status',
>>>> 'vdsmd.service') stderr:
>>>>
>>>>
>>>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.system.vdsmenv
>>>> util.connect_vdsm_json_rpc:209 {'status': {'message': 'Done', 'code': 0},
>>>> u'systemProductName': u'PowerEdge R630', u'systemSerialNumber':
>>>> u'GTH5942', u'systemFamily': u'PowerEdge', u'systemUUID':
>>>> u'4C4C4544-0054-4810-8035-C7C04F393432', u'systemManufacturer': u'Dell
>>>> Inc.'}
>>>> 2017-04-19 12:29:57 DEBUG otopi.context context.dumpEnvironment:760
>>>> ENVIRONMENT DUMP - BEGIN
>>>> 2017-04-19 12:29:57 DEBUG otopi.context context.dumpEnvironment:770 ENV
>>>> OVEHOSTED_VDSM/vdscli=_Server:'<vdsm.jsonrpcvdscli._Server object at
>>>> 0x34a8290>'
>>>> 2017-04-19 12:29:57 DEBUG otopi.context context.dumpEnvironment:774
>>>> ENVIRONMENT DUMP - END
>>>> 2017-04-19 12:29:57 DEBUG otopi.context context._executeMethod:128 Stage
>>>> late_setup METHOD otopi.plugins.gr_he_common.vm.misc.Plugin._late_setup
>>>> 2017-04-19 12:29:57 DEBUG otopi.context context._executeMethod:134
>>>> condition False
>>>> 2017-04-19 12:29:57 DEBUG otopi.context context._executeMethod:128 Stage
>>>> late_setup METHOD otopi.plugins.gr_he_setup.pki.vdsmpki.Plugin._late_setup
>>>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.pki.vdsmpki
>>>> plugin.executeRaw:813 execute: ('/bin/openssl', 'x509', '-noout', '-text',
>>>> '-in', '/etc/pki/vdsm/libvirt-spice/server-cert.pem'), executable='None',
>>>> cwd='None', env=None
>>>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.pki.vdsmpki
>>>> plugin.executeRaw:863 execute-result: ('/bin/openssl', 'x509', '-noout',
>>>> '-text', '-in', '/etc/pki/vdsm/libvirt-spice/server-cert.pem'), rc=0
>>>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.pki.vdsmpki
>>>> plugin.execute:921 execute-output: ('/bin/openssl', 'x509', '-noout',
>>>> '-text', '-in', '/etc/pki/vdsm/libvirt-spice/server-cert.pem') stdout:
>>>>
>>>> - - - - broker.log: - - - -
>>>>
>>>> Thread-2970::INFO::2017-04-19
>>>> 14:29:51,591::cpu_load_no_engine::113::cpu_load_no_engine.EngineHealth::(calculate_load)
>>>> System load total=0.0117, engine=0.0000, non-engine=0.0117
>>>> Thread-2976::INFO::2017-04-19
>>>> 14:29:56,347::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2976::INFO::2017-04-19
>>>> 14:29:56,361::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,628::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,629::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Starting submonitor ping
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,631::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Started submonitor ping, id 140603614328464
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,662::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Starting submonitor mgmt-bridge
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,667::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Started submonitor mgmt-bridge, id 140603614326992
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,688::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Starting submonitor mem-free
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,735::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Started submonitor mem-free, id 140604548742096
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,756::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Starting submonitor cpu-load-no-engine
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,798::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Started submonitor cpu-load-no-engine, id 43463568
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,841::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Starting submonitor engine-health
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:29:57,863::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>>>> Started submonitor engine-health, id 140604881915728
>>>> Thread-2983::INFO::2017-04-19
>>>> 14:29:58,153::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2984::INFO::2017-04-19
>>>> 14:30:02,179::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:30:02,249::storage_backends::119::ovirt_hosted_engine_ha.lib.storage_backends::(_check_symlinks)
>>>> Cleaning up stale LV link
>>>> '/rhev/data-center/mnt/glusterSD/sc5-gluster-1:_ovirt__engine/a81b229c-2422-4110-af45-1bc4d9a01bab/ha_agent/hosted-engine.lockspace'
>>>> Thread-2977::INFO::2017-04-19
>>>> 14:30:02,321::storage_backends::119::ovirt_hosted_engine_ha.lib.storage_backends::(_check_symlinks)
>>>> Cleaning up stale LV link
>>>> '/rhev/data-center/mnt/glusterSD/sc5-gluster-1:_ovirt__engine/a81b229c-2422-4110-af45-1bc4d9a01bab/ha_agent/hosted-engine.metadata'
>>>> Thread-2983::INFO::2017-04-19
>>>> 14:30:02,334::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2984::INFO::2017-04-19
>>>> 14:30:02,340::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2969::INFO::2017-04-19
>>>> 14:30:02,343::mem_free::50::mem_free.MemFree::(action) memFree: 254533
>>>> Thread-2980::INFO::2017-04-19
>>>> 14:30:02,382::mem_free::50::mem_free.MemFree::(action) memFree: 254530
>>>> Thread-2985::INFO::2017-04-19
>>>> 14:30:07,991::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2985::INFO::2017-04-19
>>>> 14:30:08,082::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2969::INFO::2017-04-19
>>>> 14:30:08,129::mem_free::50::mem_free.MemFree::(action) memFree: 254534
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,719::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,720::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopping submonitor ping, id 140603546975120
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,720::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopped submonitor ping, id 140603546975120
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,720::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopping submonitor mgmt-bridge, id 140603546973136
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,720::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopped submonitor mgmt-bridge, id 140603546973136
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,721::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopping submonitor mem-free, id 140603546973264
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,721::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopped submonitor mem-free, id 140603546973264
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,721::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopping submonitor cpu-load-no-engine, id 140603546976144
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,721::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopped submonitor cpu-load-no-engine, id 140603546976144
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,721::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopping submonitor engine-health, id 140603546974096
>>>> Thread-2966::INFO::2017-04-19
>>>> 14:30:09,721::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>>>> Stopped submonitor engine-health, id 140603546974096
>>>> Thread-2979::INFO::2017-04-19
>>>> 14:30:11,145::mgmt_bridge::55::mgmt_bridge.MgmtBridge::(action) Found
>>>> bridge ovirtmgmt with ports
>>>> Thread-2986::INFO::2017-04-19
>>>> 14:30:11,462::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2986::INFO::2017-04-19
>>>> 14:30:11,480::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2987::INFO::2017-04-19
>>>> 14:30:11,481::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>>>> Connection established
>>>> Thread-2987::INFO::2017-04-19
>>>> 14:30:11,488::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>>>> Connection closed
>>>> Thread-2980::INFO::2017-04-19
>>>> 14:30:11,502::mem_free::50::mem_free.MemFree::(action) memFree: 254535
>>>>
>>>> - - - - ha agent.log: - - - -
>>>>
>>>> MainThread::WARNING::2017-04-19
>>>> 14:31:16,352::agent::209::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>>>> Restarting agent, attempt '6'
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:16,383::hosted_engine::242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname)
>>>> Found certificate common name: sc5-ovirt-2.squaretrade.com
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:16,385::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
>>>> Initializing VDSM
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:18,863::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
>>>> Connecting the storage
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:18,864::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
>>>> Connecting storage server
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:23,672::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
>>>> Connecting storage server
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:23,689::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
>>>> Refreshing the storage domain
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:23,956::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
>>>> Preparing images
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:23,957::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
>>>> Preparing images
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:26,768::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
>>>> Refreshing vm.conf
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:26,769::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf)
>>>> Reloading vm.conf from the shared storage domain
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:26,769::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store)
>>>> Trying to get a fresher copy of vm configuration from the OVF_STORE
>>>> MainThread::WARNING::2017-04-19
>>>> 14:31:29,548::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
>>>> Unable to find OVF_STORE
>>>> MainThread::ERROR::2017-04-19
>>>> 14:31:29,551::config::450::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store)
>>>> Unable to identify the OVF_STORE volume, falling back to initial vm.conf.
>>>> Please ensure you already added your first data domain for regular VMs
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,576::hosted_engine::509::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
>>>> Initializing ha-broker connection
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,577::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Starting monitor ping, options {'addr': '10.181.26.1'}
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,596::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Success, id 140604881712208
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,596::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Starting monitor mgmt-bridge, options {'use_ssl': 'true', 'bridge_name':
>>>> 'ovirtmgmt', 'address': '0'}
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,600::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Success, id 140604881712016
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,600::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Starting monitor mem-free, options {'use_ssl': 'true', 'address': '0'}
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,602::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Success, id 140604881711568
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,602::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Starting monitor cpu-load-no-engine, options {'use_ssl': 'true',
>>>> 'vm_uuid': '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'}
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,605::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Success, id 140604881711312
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,605::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Starting monitor engine-health, options {'use_ssl': 'true', 'vm_uuid':
>>>> '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'}
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:29,608::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>>>> Success, id 140604881714896
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:34,326::brokerlink::179::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(set_storage_domain)
>>>> Success, id 140604881713168
>>>> MainThread::INFO::2017-04-19
>>>> 14:31:34,327::hosted_engine::601::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
>>>> Broker initialized, all submonitors started
>>>> MainThread::ERROR::2017-04-19
>>>> 14:31:34,372::agent::196::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>>>> Service sanlock is not running and the admin is responsible for starting
>>>> it. Waiting…
>>>>
>>>>
>>>> - - - - answers.conf - - - -
>>>>
>>>> [environment:default]
>>>> OVEHOSTED_CORE/rollbackProceed=none:None
>>>> OVEHOSTED_CORE/screenProceed=none:None
>>>> OVEHOSTED_CORE/deployProceed=bool:True
>>>> OVEHOSTED_CORE/upgradeProceed=none:None
>>>> OVEHOSTED_CORE/confirmSettings=bool:True
>>>> OVEHOSTED_NETWORK/fqdn=str:sc5-ovirt-engine-1.squaretrade.com
>>>> OVEHOSTED_NETWORK/bridgeName=str:ovirtmgmt
>>>> OVEHOSTED_NETWORK/firewallManager=none:None
>>>> OVEHOSTED_NETWORK/gateway=str:10.181.26.1
>>>> OVEHOSTED_ENGINE/insecureSSL=none:None
>>>> OVEHOSTED_ENGINE/clusterName=str:Default
>>>> OVEHOSTED_STORAGE/storageDatacenterName=str:hosted_datacenter
>>>> OVEHOSTED_STORAGE/domainType=str:glusterfs
>>>> OVEHOSTED_STORAGE/connectionUUID=str:c2dd62cb-79aa-4547-b191-b74e55fe5311
>>>> OVEHOSTED_STORAGE/LunID=none:None
>>>> OVEHOSTED_STORAGE/imgSizeGB=str:50
>>>> OVEHOSTED_STORAGE/mntOptions=none:None
>>>> OVEHOSTED_STORAGE/iSCSIPortalIPAddress=none:None
>>>> OVEHOSTED_STORAGE/metadataVolumeUUID=str:ac9d7fbf-fb41-4b96-8940-b95698e58b18
>>>> OVEHOSTED_STORAGE/sdUUID=str:a81b229c-2422-4110-af45-1bc4d9a01bab
>>>> OVEHOSTED_STORAGE/iSCSITargetName=none:None
>>>> OVEHOSTED_STORAGE/metadataImageUUID=str:298fef26-3039-4c9e-bee8-39770fb2e821
>>>> OVEHOSTED_STORAGE/lockspaceVolumeUUID=str:c90a1959-369d-4109-8a6e-5c81f735e43b
>>>> OVEHOSTED_STORAGE/iSCSIPortalPort=none:None
>>>> OVEHOSTED_STORAGE/imgUUID=str:284c61f2-a901-46f0-a05a-2e5bf220efc6
>>>> OVEHOSTED_STORAGE/confImageUUID=str:8bcc7fae-2fbc-40aa-b75a-6c6adb8ba7dc
>>>> OVEHOSTED_STORAGE/spUUID=str:00000000-0000-0000-0000-000000000000
>>>> OVEHOSTED_STORAGE/lockspaceImageUUID=str:0ce0f2f9-b9ea-45b3-b091-d4f0e46e150f
>>>> OVEHOSTED_ENGINE/enableHcGlusterService=bool:False
>>>> OVEHOSTED_STORAGE/storageDomainName=str:hosted_storage
>>>> OVEHOSTED_STORAGE/iSCSIPortal=none:None
>>>> OVEHOSTED_STORAGE/volUUID=str:16304df2-2d95-4d75-932d-21d7c6de000d
>>>> OVEHOSTED_STORAGE/vgUUID=none:None
>>>> OVEHOSTED_STORAGE/confVolUUID=str:3b42f578-5d38-4820-b977-fa37b49926f7
>>>> OVEHOSTED_STORAGE/storageDomainConnection=str:sc5-gluster-1:/ovirt_engine
>>>> OVEHOSTED_STORAGE/iSCSIPortalUser=none:None
>>>> OVEHOSTED_VDSM/consoleType=str:qxl
>>>> OVEHOSTED_VM/vmMemSizeMB=int:16384
>>>> OVEHOSTED_VM/vmUUID=str:04ff4cf1-135a-4918-9a1f-8023322f89a3
>>>> OVEHOSTED_VM/vmMACAddr=str:00:16:3e:6e:86:b2
>>>> OVEHOSTED_VM/emulatedMachine=str:pc
>>>> OVEHOSTED_VM/consoleUUID=str:7d26f23e-59b6-4378-a0a4-f50e9653ac78
>>>> OVEHOSTED_VM/vmVCpus=str:4
>>>> OVEHOSTED_VM/nicUUID=str:a2b7dc56-02f6-4585-bd50-079b17aafe1f
>>>> OVEHOSTED_VM/cdromUUID=str:cde48946-a7dc-41a5-87ee-854eb4d4dc22
>>>> OVEHOSTED_VM/ovfArchive=str:/usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.1-20170406.1.el7.centos.ova
>>>> OVEHOSTED_VM/vmCDRom=none:None
>>>> OVEHOSTED_VM/automateVMShutdown=bool:True
>>>> OVEHOSTED_VM/cloudInitISO=str:generate
>>>> OVEHOSTED_VM/cloudinitInstanceDomainName=str:squaretrade.com
>>>> OVEHOSTED_VM/cloudinitInstanceHostName=str:sc5-ovirt-engine-1.squaretrade.com
>>>> OVEHOSTED_VM/rootSshPubkey=str:ssh-rsa [snip]
>>>> OVEHOSTED_VM/cloudinitExecuteEngineSetup=bool:True
>>>> OVEHOSTED_VM/cloudinitVMStaticCIDR=[snip]
>>>> OVEHOSTED_VM/cloudinitVMTZ=str:America/Los_Angeles
>>>> OVEHOSTED_VM/rootSshAccess=str:yes
>>>> OVEHOSTED_VM/cloudinitVMETCHOSTS=bool:False
>>>> OVEHOSTED_VM/cloudinitVMDNS=[snip]
>>>> OVEHOSTED_VDSM/spicePkiSubject=str:O=squaretrade.com,
>>>> CN=sc5-ovirt-2.squaretrade.com
>>>> OVEHOSTED_VDSM/pkiSubject=str:/C=EN/L=Test/O=Test/CN=Test
>>>> OVEHOSTED_VDSM/caSubject=str:/C=EN/L=Test/O=Test/CN=TestCA
>>>> OVEHOSTED_VDSM/cpu=str:model_Haswell-noTSX
>>>> OVEHOSTED_NOTIF/smtpPort=str:25
>>>> OVEHOSTED_NOTIF/smtpServer=[snip]
>>>> OVEHOSTED_NOTIF/sourceEmail=[snip]
>>>> OVEHOSTED_NOTIF/destEmail=[snip]
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users@ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>
>
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
