> On Apr 19, 2017, at 11:35 PM, knarra <kna...@redhat.com> wrote:
>
> On 04/20/2017 03:15 AM, Jamie Lawrence wrote:
>> I trialed installing the hosted engine, following the instructions at
>> http://www.ovirt.org/documentation/self-hosted/chap-Deploying_Self-Hosted_Engine/
>> . This is using Gluster as the backend storage subsystem.
>>
>> Answer file at the end.
>>
>> Per the docs,
>>
>> "When the hosted-engine deployment script completes successfully, the oVirt
>> Engine is configured and running on your host. The Engine has already
>> configured the data center, cluster, host, the Engine virtual machine, and a
>> shared storage domain dedicated to the Engine virtual machine.”
>>
>> In my case, this is false. The installation claims success, but the hosted
>> engine VM stays stopped, unless I start it manually.
> During the install process there is a step where HE vm is stopped and
> started. Can you check if this has happened correctly ?
The installer claimed it did, but I believe it didn’t. Below the error from my
original email, there’s the below (apologies for not including it earlier; I
missed it). Note: 04ff4cf1-135a-4918-9a1f-8023322f89a3 is the HE - I’m pretty
sure it is complaining about itself. (In any case, I verified that there are no
other VMs running with both virsh and vdsClient.)
2017-04-19 12:27:02 DEBUG otopi.context context._executeMethod:128 Stage
late_setup METHOD otopi.plugins.gr_he_setup.vm.runvm.Plugin._late_setup
2017-04-19 12:27:02 DEBUG otopi.plugins.gr_he_setup.vm.runvm
runvm._late_setup:83 {'status': {'message': 'Done', 'code': 0}, 'items':
[u'04ff4cf1-135a-4918-9a1f-8023322f89a3']}
2017-04-19 12:27:02 ERROR otopi.plugins.gr_he_setup.vm.runvm
runvm._late_setup:91 The following VMs have been found:
04ff4cf1-135a-4918-9a1f-8023322f89a3
2017-04-19 12:27:02 DEBUG otopi.context context._executeMethod:142 method
exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in
_executeMethod
method['method']()
File
"/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/gr-he-setup/vm/runvm.py",
line 95, in _late_setup
_('Cannot setup Hosted Engine with other VMs running')
RuntimeError: Cannot setup Hosted Engine with other VMs running
2017-04-19 12:27:02 ERROR otopi.context context._executeMethod:151 Failed to
execute stage 'Environment setup': Cannot setup Hosted Engine with other VMs
running
2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:760 ENVIRONMENT
DUMP - BEGIN
2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:770 ENV
BASE/error=bool:'True'
2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:770 ENV
BASE/exceptionInfo=list:'[(<type 'exceptions.RuntimeError'>,
RuntimeError('Cannot setup Hosted Engine with other VMs running',), <traceback
object at 0x4358098>)]'
2017-04-19 12:27:02 DEBUG otopi.context context.dumpEnvironment:774 ENVIRONMENT
DUMP - END
>> If I start it manually, the default DC is down, the default cluster has the
>> installation host in the cluster, there is no storage, and the VM doesn’t
>> show up in the GUI. In this install run, I have not yet started the engine
>> manually.
> you wont be seeing HE vm until HE storage is imported into the UI. HE storage
> will be automatically imported into the UI (which will import HE vm too )once
> a master domain is present .
Sure; I’m just attempting to provide context.
>> I assume this is related to the errors in ovirt-hosted-engine-setup.log,
>> below. (The timestamps are confusing; it looks like the Python errors are
>> logged some time after they’re captured or something.) The HA broker and
>> agent logs just show them looping in the sequence below.
>>
>> Is there a decent way to pick this up and continue? If not, how do I make
>> this work?
> Can you please check the following things.
>
> 1) is glusterd running on all the nodes ? 'systemctl status glistered’
> 2) Are you able to connect to your storage server which is ovirt_engine in
> your case.
> 3) Can you check if all the brick process in the volume is up ?
1) Verified that glusterd is running on all three nodes.
2)
[root@sc5-thing-1]# mount -tglusterfs sc5-gluster-1:/ovirt_engine
/mnt/ovirt_engine
[root@sc5-thing-1]# df -h
Filesystem Size Used Avail Use% Mounted on
[…]
sc5-gluster-1:/ovirt_engine 300G 2.6G 298G 1% /mnt/ovirt_engine
3)
[root@sc5-gluster-1 jlawrence]# gluster volume status
Status of volume: ovirt_engine
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick sc5-gluster-1:/gluster-bricks/ovirt_e
ngine/ovirt_engine-1 49217 0 Y 22102
Brick sc5-gluster-2:/gluster-bricks/ovirt_e
ngine/ovirt_engine-1 49157 0 Y 37842
Brick sc5-gluster-3:/gluster-bricks/ovirt_e
ngine/ovirt_engine-1 49157 0 Y 112018
Self-heal Daemon on localhost N/A N/A Y 113364
Self-heal Daemon on sc5-gluster-2.squaretra
de.com N/A N/A Y 19344
Self-heal Daemon on sc5-gluster-3.squaretra
de.com N/A N/A Y 37956
Task Status of Volume ovirt_engine
------------------------------------------------------------------------------
There are no active volume tasks
Volume ovirt_images is not started
> Thanks
> kasturi.
Thank you!
-j
>>
>> Thanks,
>>
>> -j
>>
>> - - - - ovirt-hosted-engine-setup.log snippet: - - - -
>>
>> 2017-04-19 12:29:55 DEBUG otopi.context context._executeMethod:128 Stage
>> late_setup METHOD otopi.plugins.gr_he_setup.system.vdsmenv.Plugin._late_setup
>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>> systemd.status:90 check service vdsmd status
>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>> plugin.executeRaw:813 execute: ('/bin/systemctl', 'status',
>> 'vdsmd.service'), executable='None', cwd='None', env=None
>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>> plugin.executeRaw:863 execute-result: ('/bin/systemctl', 'status',
>> 'vdsmd.service'), rc=0
>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>> plugin.execute:921 execute-output: ('/bin/systemctl', 'status',
>> 'vdsmd.service') stdout:
>> ● vdsmd.service - Virtual Desktop Server Manager
>> Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor
>> preset: enabled)
>> Active: active (running) since Wed 2017-04-19 12:26:59 PDT; 2min 55s ago
>> Process: 67370 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh
>> --post-stop (code=exited, status=0/SUCCESS)
>> Process: 69995 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh
>> --pre-start (code=exited, status=0/SUCCESS)
>> Main PID: 70062 (vdsm)
>> CGroup: /system.slice/vdsmd.service
>> └─70062 /usr/bin/python2 /usr/share/vdsm/vdsm
>>
>> Apr 19 12:29:00 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to
>> broker, the number of errors has exceeded the limit (1)
>> Apr 19 12:29:00 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>> failed to retrieve Hosted Engine HA info
>> Traceback (most
>> recent call last):
>> File
>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in _getHaInfo
>> stats =
>> instance.get_all_stats()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>> line 102, in get_all_stats
>> with
>> broker.connection(self._retries, self._wait):
>> File
>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>> return
>> self.gen.next()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 99, in connection
>>
>> self.connect(retries, wait)
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 78, in connect
>> raise
>> BrokerConnectionError(error_msg)
>>
>> BrokerConnectionError: Failed to connect to broker, the number of errors has
>> exceeded the limit (1)
>> Apr 19 12:29:15 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to
>> broker, the number of errors has exceeded the limit (1)
>> Apr 19 12:29:15 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>> failed to retrieve Hosted Engine HA info
>> Traceback (most
>> recent call last):
>> File
>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in _getHaInfo
>> stats =
>> instance.get_all_stats()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>> line 102, in get_all_stats
>> with
>> broker.connection(self._retries, self._wait):
>> File
>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>> return
>> self.gen.next()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 99, in connection
>>
>> self.connect(retries, wait)
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 78, in connect
>> raise
>> BrokerConnectionError(error_msg)
>>
>> BrokerConnectionError: Failed to connect to broker, the number of errors has
>> exceeded the limit (1)
>> Apr 19 12:29:20 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root WARN
>> File:
>> /var/lib/libvirt/qemu/channels/04ff4cf1-135a-4918-9a1f-8023322f89a3.com.redhat.rhevm.vdsm
>> already removed
>> Apr 19 12:29:20 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root WARN
>> File:
>> /var/lib/libvirt/qemu/channels/04ff4cf1-135a-4918-9a1f-8023322f89a3.org.qemu.guest_agent.0
>> already removed
>> Apr 19 12:29:30 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to
>> broker, the number of errors has exceeded the limit (1)
>> Apr 19 12:29:30 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>> failed to retrieve Hosted Engine HA info
>> Traceback (most
>> recent call last):
>> File
>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in _getHaInfo
>> stats =
>> instance.get_all_stats()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>> line 102, in get_all_stats
>> with
>> broker.connection(self._retries, self._wait):
>> File
>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>> return
>> self.gen.next()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 99, in connection
>>
>> self.connect(retries, wait)
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 78, in connect
>> raise
>> BrokerConnectionError(error_msg)
>>
>> BrokerConnectionError: Failed to connect to broker, the number of errors has
>> exceeded the limit (1)
>> Apr 19 12:29:45 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm
>> ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink ERROR Failed to connect to
>> broker, the number of errors has exceeded the limit (1)
>> Apr 19 12:29:45 sc5-ovirt-2.squaretrade.com vdsm[70062]: vdsm root ERROR
>> failed to retrieve Hosted Engine HA info
>> Traceback (most
>> recent call last):
>> File
>> "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 231, in _getHaInfo
>> stats =
>> instance.get_all_stats()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
>> line 102, in get_all_stats
>> with
>> broker.connection(self._retries, self._wait):
>> File
>> "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
>> return
>> self.gen.next()
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 99, in connection
>>
>> self.connect(retries, wait)
>> File
>> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
>> line 78, in connect
>> raise
>> BrokerConnectionError(error_msg)
>>
>> BrokerConnectionError: Failed to connect to broker, the number of errors has
>> exceeded the limit (1)
>>
>> 2017-04-19 12:29:55 DEBUG otopi.plugins.otopi.services.systemd
>> plugin.execute:926 execute-output: ('/bin/systemctl', 'status',
>> 'vdsmd.service') stderr:
>>
>>
>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.system.vdsmenv
>> util.connect_vdsm_json_rpc:209 {'status': {'message': 'Done', 'code': 0},
>> u'systemProductName': u'PowerEdge R630', u'systemSerialNumber': u'GTH5942',
>> u'systemFamily': u'PowerEdge', u'systemUUID':
>> u'4C4C4544-0054-4810-8035-C7C04F393432', u'systemManufacturer': u'Dell Inc.'}
>> 2017-04-19 12:29:57 DEBUG otopi.context context.dumpEnvironment:760
>> ENVIRONMENT DUMP - BEGIN
>> 2017-04-19 12:29:57 DEBUG otopi.context context.dumpEnvironment:770 ENV
>> OVEHOSTED_VDSM/vdscli=_Server:'<vdsm.jsonrpcvdscli._Server object at
>> 0x34a8290>'
>> 2017-04-19 12:29:57 DEBUG otopi.context context.dumpEnvironment:774
>> ENVIRONMENT DUMP - END
>> 2017-04-19 12:29:57 DEBUG otopi.context context._executeMethod:128 Stage
>> late_setup METHOD otopi.plugins.gr_he_common.vm.misc.Plugin._late_setup
>> 2017-04-19 12:29:57 DEBUG otopi.context context._executeMethod:134 condition
>> False
>> 2017-04-19 12:29:57 DEBUG otopi.context context._executeMethod:128 Stage
>> late_setup METHOD otopi.plugins.gr_he_setup.pki.vdsmpki.Plugin._late_setup
>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.pki.vdsmpki
>> plugin.executeRaw:813 execute: ('/bin/openssl', 'x509', '-noout', '-text',
>> '-in', '/etc/pki/vdsm/libvirt-spice/server-cert.pem'), executable='None',
>> cwd='None', env=None
>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.pki.vdsmpki
>> plugin.executeRaw:863 execute-result: ('/bin/openssl', 'x509', '-noout',
>> '-text', '-in', '/etc/pki/vdsm/libvirt-spice/server-cert.pem'), rc=0
>> 2017-04-19 12:29:57 DEBUG otopi.plugins.gr_he_setup.pki.vdsmpki
>> plugin.execute:921 execute-output: ('/bin/openssl', 'x509', '-noout',
>> '-text', '-in', '/etc/pki/vdsm/libvirt-spice/server-cert.pem') stdout:
>>
>> - - - - broker.log: - - - -
>>
>> Thread-2970::INFO::2017-04-19
>> 14:29:51,591::cpu_load_no_engine::113::cpu_load_no_engine.EngineHealth::(calculate_load)
>> System load total=0.0117, engine=0.0000, non-engine=0.0117
>> Thread-2976::INFO::2017-04-19
>> 14:29:56,347::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2976::INFO::2017-04-19
>> 14:29:56,361::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,628::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,629::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Starting submonitor ping
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,631::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Started submonitor ping, id 140603614328464
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,662::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Starting submonitor mgmt-bridge
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,667::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Started submonitor mgmt-bridge, id 140603614326992
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,688::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Starting submonitor mem-free
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,735::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Started submonitor mem-free, id 140604548742096
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,756::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Starting submonitor cpu-load-no-engine
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,798::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Started submonitor cpu-load-no-engine, id 43463568
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,841::monitor::66::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Starting submonitor engine-health
>> Thread-2977::INFO::2017-04-19
>> 14:29:57,863::monitor::79::ovirt_hosted_engine_ha.broker.monitor.Monitor::(start_submonitor)
>> Started submonitor engine-health, id 140604881915728
>> Thread-2983::INFO::2017-04-19
>> 14:29:58,153::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2984::INFO::2017-04-19
>> 14:30:02,179::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2977::INFO::2017-04-19
>> 14:30:02,249::storage_backends::119::ovirt_hosted_engine_ha.lib.storage_backends::(_check_symlinks)
>> Cleaning up stale LV link
>> '/rhev/data-center/mnt/glusterSD/sc5-gluster-1:_ovirt__engine/a81b229c-2422-4110-af45-1bc4d9a01bab/ha_agent/hosted-engine.lockspace'
>> Thread-2977::INFO::2017-04-19
>> 14:30:02,321::storage_backends::119::ovirt_hosted_engine_ha.lib.storage_backends::(_check_symlinks)
>> Cleaning up stale LV link
>> '/rhev/data-center/mnt/glusterSD/sc5-gluster-1:_ovirt__engine/a81b229c-2422-4110-af45-1bc4d9a01bab/ha_agent/hosted-engine.metadata'
>> Thread-2983::INFO::2017-04-19
>> 14:30:02,334::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2984::INFO::2017-04-19
>> 14:30:02,340::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2969::INFO::2017-04-19
>> 14:30:02,343::mem_free::50::mem_free.MemFree::(action) memFree: 254533
>> Thread-2980::INFO::2017-04-19
>> 14:30:02,382::mem_free::50::mem_free.MemFree::(action) memFree: 254530
>> Thread-2985::INFO::2017-04-19
>> 14:30:07,991::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2985::INFO::2017-04-19
>> 14:30:08,082::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2969::INFO::2017-04-19
>> 14:30:08,129::mem_free::50::mem_free.MemFree::(action) memFree: 254534
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,719::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,720::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopping submonitor ping, id 140603546975120
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,720::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopped submonitor ping, id 140603546975120
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,720::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopping submonitor mgmt-bridge, id 140603546973136
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,720::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopped submonitor mgmt-bridge, id 140603546973136
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,721::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopping submonitor mem-free, id 140603546973264
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,721::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopped submonitor mem-free, id 140603546973264
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,721::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopping submonitor cpu-load-no-engine, id 140603546976144
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,721::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopped submonitor cpu-load-no-engine, id 140603546976144
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,721::monitor::90::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopping submonitor engine-health, id 140603546974096
>> Thread-2966::INFO::2017-04-19
>> 14:30:09,721::monitor::99::ovirt_hosted_engine_ha.broker.monitor.Monitor::(stop_submonitor)
>> Stopped submonitor engine-health, id 140603546974096
>> Thread-2979::INFO::2017-04-19
>> 14:30:11,145::mgmt_bridge::55::mgmt_bridge.MgmtBridge::(action) Found bridge
>> ovirtmgmt with ports
>> Thread-2986::INFO::2017-04-19
>> 14:30:11,462::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2986::INFO::2017-04-19
>> 14:30:11,480::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2987::INFO::2017-04-19
>> 14:30:11,481::listener::134::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(setup)
>> Connection established
>> Thread-2987::INFO::2017-04-19
>> 14:30:11,488::listener::186::ovirt_hosted_engine_ha.broker.listener.ConnectionHandler::(handle)
>> Connection closed
>> Thread-2980::INFO::2017-04-19
>> 14:30:11,502::mem_free::50::mem_free.MemFree::(action) memFree: 254535
>>
>> - - - - ha agent.log: - - - -
>>
>> MainThread::WARNING::2017-04-19
>> 14:31:16,352::agent::209::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>> Restarting agent, attempt '6'
>> MainThread::INFO::2017-04-19
>> 14:31:16,383::hosted_engine::242::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname)
>> Found certificate common name: sc5-ovirt-2.squaretrade.com
>> MainThread::INFO::2017-04-19
>> 14:31:16,385::hosted_engine::604::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
>> Initializing VDSM
>> MainThread::INFO::2017-04-19
>> 14:31:18,863::hosted_engine::630::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
>> Connecting the storage
>> MainThread::INFO::2017-04-19
>> 14:31:18,864::storage_server::219::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
>> Connecting storage server
>> MainThread::INFO::2017-04-19
>> 14:31:23,672::storage_server::226::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
>> Connecting storage server
>> MainThread::INFO::2017-04-19
>> 14:31:23,689::storage_server::233::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
>> Refreshing the storage domain
>> MainThread::INFO::2017-04-19
>> 14:31:23,956::hosted_engine::657::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
>> Preparing images
>> MainThread::INFO::2017-04-19
>> 14:31:23,957::image::126::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
>> Preparing images
>> MainThread::INFO::2017-04-19
>> 14:31:26,768::hosted_engine::660::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
>> Refreshing vm.conf
>> MainThread::INFO::2017-04-19
>> 14:31:26,769::config::485::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_vm_conf)
>> Reloading vm.conf from the shared storage domain
>> MainThread::INFO::2017-04-19
>> 14:31:26,769::config::412::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store)
>> Trying to get a fresher copy of vm configuration from the OVF_STORE
>> MainThread::WARNING::2017-04-19
>> 14:31:29,548::ovf_store::107::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan)
>> Unable to find OVF_STORE
>> MainThread::ERROR::2017-04-19
>> 14:31:29,551::config::450::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(_get_vm_conf_content_from_ovf_store)
>> Unable to identify the OVF_STORE volume, falling back to initial vm.conf.
>> Please ensure you already added your first data domain for regular VMs
>> MainThread::INFO::2017-04-19
>> 14:31:29,576::hosted_engine::509::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
>> Initializing ha-broker connection
>> MainThread::INFO::2017-04-19
>> 14:31:29,577::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Starting monitor ping, options {'addr': '10.181.26.1'}
>> MainThread::INFO::2017-04-19
>> 14:31:29,596::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Success, id 140604881712208
>> MainThread::INFO::2017-04-19
>> 14:31:29,596::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Starting monitor mgmt-bridge, options {'use_ssl': 'true', 'bridge_name':
>> 'ovirtmgmt', 'address': '0'}
>> MainThread::INFO::2017-04-19
>> 14:31:29,600::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Success, id 140604881712016
>> MainThread::INFO::2017-04-19
>> 14:31:29,600::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Starting monitor mem-free, options {'use_ssl': 'true', 'address': '0'}
>> MainThread::INFO::2017-04-19
>> 14:31:29,602::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Success, id 140604881711568
>> MainThread::INFO::2017-04-19
>> 14:31:29,602::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Starting monitor cpu-load-no-engine, options {'use_ssl': 'true', 'vm_uuid':
>> '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'}
>> MainThread::INFO::2017-04-19
>> 14:31:29,605::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Success, id 140604881711312
>> MainThread::INFO::2017-04-19
>> 14:31:29,605::brokerlink::130::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Starting monitor engine-health, options {'use_ssl': 'true', 'vm_uuid':
>> '04ff4cf1-135a-4918-9a1f-8023322f89a3', 'address': '0'}
>> MainThread::INFO::2017-04-19
>> 14:31:29,608::brokerlink::141::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor)
>> Success, id 140604881714896
>> MainThread::INFO::2017-04-19
>> 14:31:34,326::brokerlink::179::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(set_storage_domain)
>> Success, id 140604881713168
>> MainThread::INFO::2017-04-19
>> 14:31:34,327::hosted_engine::601::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_broker)
>> Broker initialized, all submonitors started
>> MainThread::ERROR::2017-04-19
>> 14:31:34,372::agent::196::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>> Service sanlock is not running and the admin is responsible for starting
>> it. Waiting…
>>
>>
>> - - - - answers.conf - - - -
>>
>> [environment:default]
>> OVEHOSTED_CORE/rollbackProceed=none:None
>> OVEHOSTED_CORE/screenProceed=none:None
>> OVEHOSTED_CORE/deployProceed=bool:True
>> OVEHOSTED_CORE/upgradeProceed=none:None
>> OVEHOSTED_CORE/confirmSettings=bool:True
>> OVEHOSTED_NETWORK/fqdn=str:sc5-ovirt-engine-1.squaretrade.com
>> OVEHOSTED_NETWORK/bridgeName=str:ovirtmgmt
>> OVEHOSTED_NETWORK/firewallManager=none:None
>> OVEHOSTED_NETWORK/gateway=str:10.181.26.1
>> OVEHOSTED_ENGINE/insecureSSL=none:None
>> OVEHOSTED_ENGINE/clusterName=str:Default
>> OVEHOSTED_STORAGE/storageDatacenterName=str:hosted_datacenter
>> OVEHOSTED_STORAGE/domainType=str:glusterfs
>> OVEHOSTED_STORAGE/connectionUUID=str:c2dd62cb-79aa-4547-b191-b74e55fe5311
>> OVEHOSTED_STORAGE/LunID=none:None
>> OVEHOSTED_STORAGE/imgSizeGB=str:50
>> OVEHOSTED_STORAGE/mntOptions=none:None
>> OVEHOSTED_STORAGE/iSCSIPortalIPAddress=none:None
>> OVEHOSTED_STORAGE/metadataVolumeUUID=str:ac9d7fbf-fb41-4b96-8940-b95698e58b18
>> OVEHOSTED_STORAGE/sdUUID=str:a81b229c-2422-4110-af45-1bc4d9a01bab
>> OVEHOSTED_STORAGE/iSCSITargetName=none:None
>> OVEHOSTED_STORAGE/metadataImageUUID=str:298fef26-3039-4c9e-bee8-39770fb2e821
>> OVEHOSTED_STORAGE/lockspaceVolumeUUID=str:c90a1959-369d-4109-8a6e-5c81f735e43b
>> OVEHOSTED_STORAGE/iSCSIPortalPort=none:None
>> OVEHOSTED_STORAGE/imgUUID=str:284c61f2-a901-46f0-a05a-2e5bf220efc6
>> OVEHOSTED_STORAGE/confImageUUID=str:8bcc7fae-2fbc-40aa-b75a-6c6adb8ba7dc
>> OVEHOSTED_STORAGE/spUUID=str:00000000-0000-0000-0000-000000000000
>> OVEHOSTED_STORAGE/lockspaceImageUUID=str:0ce0f2f9-b9ea-45b3-b091-d4f0e46e150f
>> OVEHOSTED_ENGINE/enableHcGlusterService=bool:False
>> OVEHOSTED_STORAGE/storageDomainName=str:hosted_storage
>> OVEHOSTED_STORAGE/iSCSIPortal=none:None
>> OVEHOSTED_STORAGE/volUUID=str:16304df2-2d95-4d75-932d-21d7c6de000d
>> OVEHOSTED_STORAGE/vgUUID=none:None
>> OVEHOSTED_STORAGE/confVolUUID=str:3b42f578-5d38-4820-b977-fa37b49926f7
>> OVEHOSTED_STORAGE/storageDomainConnection=str:sc5-gluster-1:/ovirt_engine
>> OVEHOSTED_STORAGE/iSCSIPortalUser=none:None
>> OVEHOSTED_VDSM/consoleType=str:qxl
>> OVEHOSTED_VM/vmMemSizeMB=int:16384
>> OVEHOSTED_VM/vmUUID=str:04ff4cf1-135a-4918-9a1f-8023322f89a3
>> OVEHOSTED_VM/vmMACAddr=str:00:16:3e:6e:86:b2
>> OVEHOSTED_VM/emulatedMachine=str:pc
>> OVEHOSTED_VM/consoleUUID=str:7d26f23e-59b6-4378-a0a4-f50e9653ac78
>> OVEHOSTED_VM/vmVCpus=str:4
>> OVEHOSTED_VM/nicUUID=str:a2b7dc56-02f6-4585-bd50-079b17aafe1f
>> OVEHOSTED_VM/cdromUUID=str:cde48946-a7dc-41a5-87ee-854eb4d4dc22
>> OVEHOSTED_VM/ovfArchive=str:/usr/share/ovirt-engine-appliance/ovirt-engine-appliance-4.1-20170406.1.el7.centos.ova
>> OVEHOSTED_VM/vmCDRom=none:None
>> OVEHOSTED_VM/automateVMShutdown=bool:True
>> OVEHOSTED_VM/cloudInitISO=str:generate
>> OVEHOSTED_VM/cloudinitInstanceDomainName=str:squaretrade.com
>> OVEHOSTED_VM/cloudinitInstanceHostName=str:sc5-ovirt-engine-1.squaretrade.com
>> OVEHOSTED_VM/rootSshPubkey=str:ssh-rsa [snip]
>> OVEHOSTED_VM/cloudinitExecuteEngineSetup=bool:True
>> OVEHOSTED_VM/cloudinitVMStaticCIDR=[snip]
>> OVEHOSTED_VM/cloudinitVMTZ=str:America/Los_Angeles
>> OVEHOSTED_VM/rootSshAccess=str:yes
>> OVEHOSTED_VM/cloudinitVMETCHOSTS=bool:False
>> OVEHOSTED_VM/cloudinitVMDNS=[snip]
>> OVEHOSTED_VDSM/spicePkiSubject=str:O=squaretrade.com,
>> CN=sc5-ovirt-2.squaretrade.com
>> OVEHOSTED_VDSM/pkiSubject=str:/C=EN/L=Test/O=Test/CN=Test
>> OVEHOSTED_VDSM/caSubject=str:/C=EN/L=Test/O=Test/CN=TestCA
>> OVEHOSTED_VDSM/cpu=str:model_Haswell-noTSX
>> OVEHOSTED_NOTIF/smtpPort=str:25
>> OVEHOSTED_NOTIF/smtpServer=[snip]
>> OVEHOSTED_NOTIF/sourceEmail=[snip]
>> OVEHOSTED_NOTIF/destEmail=[snip]
>> _______________________________________________
>> Users mailing list
>> Users@ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
