14.12.2018 15:52, Martin Sivak пишет:
Hi,

 > Host id is not set

This is an internal error report that should normally not happen. It means the ovirt-ha-agent asked for a storage operation before it registered itself with the broker. If this happens seldom then it looks like a race condition.

I'm seeing this events also on my setup randomly and seldom.
versions:
ovirt-release42-4.2.7.1-1.el7.noarch
ovirt-hosted-engine-ha-2.2.18-1.el7.noarch
ovirt-host-4.2.3-1.el7.x86_64
...
I have emails from 15 nov, 21 nov, 01 dec, 05 dec...
Last event at 09 dec 16:35:

-----------------
MainThread::INFO::2018-12-09 16:34:05,223::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400) MainThread::INFO::2018-12-09 16:34:15,360::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
....... ALL OK HERE UNTIL......
MainThread::ERROR::2018-12-09 16:34:25,373::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring) Unhandled monitoring loop exception
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py", line 428, in start_monitoring
    self._monitoring_loop()
.....
File "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 135, in get_stats_from_storage
    result = self._proxy.get_stats()
.....
error: [Errno 2] No such file or directory
.....
-------------------------
MainThread::ERROR::2018-12-09 16:34:25,380::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Trying to restart agent
.....
MainThread::INFO::2018-12-09 16:34:46,353::brokerlink::77::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Starting monitor storage-domain, options {'sd_uuid': '0571ac7b-a28e-4e20-9cd8-4803e40ec602'} MainThread::INFO::2018-12-09 16:34:46,354::brokerlink::85::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(start_monitor) Success, id storage-domain
.....
MainThread::INFO::2018-12-09 16:35:25,062::brokerlink::68::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStarting-EngineUp) sent? sent
.....
MainThread::INFO::2018-12-09 16:35:31,209::hosted_engine::491::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineUp (score: 3400)
-----------------------------


I would recommend opening a bug report with all the logs we talked about and all the RPM versions (ovirt-hosted-engine-ha and ovirt-hosted-engine-setup packages). Use this link to go directly to the right component: https://bugzilla.redhat.com/enter_bug.cgi?product=ovirt-hosted-engine-ha

Are anybody open this bug report ? I do not have account in RH bugzilla..

(Accidentally this (9 dec) log file is rotated while i wrote this email...)



On Fri, Dec 14, 2018 at 12:35 PM fsoyer <fso...@systea.fr <mailto:fso...@systea.fr>> wrote:

    In borker.log I found this, just before 05:59am:

        Thread-3::INFO::2018-12-13
        05:58:45,634::mem_free::51::mem_free.MemFree::(action) memFree:
        82101
        Thread-1::INFO::2018-12-13
        05:58:46,322::ping::60::ping.Ping::(action) Successfully pinged
        10.0.1.254
        Thread-5::INFO::2018-12-13
        
05:58:46,611::engine_health::241::engine_health.EngineHealth::(_result_from_stats)
        VM is up on this host with healthy engine
        Thread-2::INFO::2018-12-13
        05:58:49,144::mgmt_bridge::62::mgmt_bridge.MgmtBridge::(action)
        Found bridge ovirtmgmt with ports
        StatusStorageThread::ERROR::2018-12-13
        
05:58:54,935::status_broker::90::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(run)
        Failed to update state.
        Traceback (most recent call last):
           File
        
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
        line 82, in run
             if (self._status_broker._inquire_whiteboard_lock() or
           File
        
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
        line 190, in _inquire_whiteboard_lock
             self.host_id, self._lease_file)
           File
        
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/status_broker.py",
        line 128, in host_id
             raise ex.HostIdNotLockedError("Host id is not set")
        HostIdNotLockedError: Host id is not set
        StatusStorageThread::ERROR::2018-12-13
        
05:58:54,937::status_broker::70::ovirt_hosted_engine_ha.broker.status_broker.StatusBroker.Update::(trigger_restart)
        Trying to restart the broker

    "Host is not set" ???
    --

    Regards,

    *Frank*



    Le Vendredi, Décembre 14, 2018 12:27 CET, Martin Sivak
    <msi...@redhat.com <mailto:msi...@redhat.com>> a écrit:
    Hi,
    check the broker.log as well. The connect is used to talk to
    ovirt-ha-broker service socket.
    Best regards
    Martin Sivak
    On Fri, Dec 14, 2018 at 12:20 PM fsoyer <fso...@systea.fr
    <mailto:fso...@systea.fr>> wrote:

        I think I have it in agent.log. What can be this "file not
        found" ?

            MainThread::ERROR::2018-12-13
            
05:59:03,909::hosted_engine::431::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
            Unhandled monitoring loop exception
            Traceback (most recent call last):
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
            line 428, in start_monitoring
                self._monitoring_loop()
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
            line 447, in _monitoring_loop
                for old_state, state, delay in self.fsm:
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/fsm/machine.py",
            line 127, in next
                new_data = self.refresh(self._state.data)
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/state_machine.py",
            line 81, in refresh
                stats.update(self.hosted_engine.collect_stats())
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
            line 736, in collect_stats
                all_stats = self._broker.get_stats_from_storage()
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
            line 135, in get_stats_from_storage
                result = self._proxy.get_stats()
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in
            __call__
                return self.__send(self.__name, args)
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in
            __request
                verbose=self.__verbose
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in
            request
                return self.single_request(host, handler,
            request_body, verbose)
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in
            single_request
                self.send_content(h, request_body)
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in
            send_content
                connection.endheaders(request_body)
              File "/usr/lib64/python2.7/httplib.py", line 1037, in
            endheaders
                self._send_output(message_body)
              File "/usr/lib64/python2.7/httplib.py", line 881, in
            _send_output
                self.send(msg)
              File "/usr/lib64/python2.7/httplib.py", line 843, in send
                self.connect()
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py",
            line 52, in connect
                self.sock.connect(base64.b16decode(self.host))
              File "/usr/lib64/python2.7/socket.py", line 224, in meth
                return getattr(self._sock,name)(*args)
            error: [Errno 2] No such file or directory
            MainThread::ERROR::2018-12-13
            
05:59:04,043::agent::144::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
            Traceback (most recent call last):
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
            line 131, in _run_agent
                return action(he)
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/agent.py",
            line 55, in action_proper
                return he.start_monitoring()
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
            line 435, in start_monitoring
                self.publish(stopped)
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
            line 337, in publish
                self._push_to_storage(blocks)
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/agent/hosted_engine.py",
            line 707, in _push_to_storage
                self._broker.put_stats_on_storage(self.host_id, blocks)
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
            line 105, in put_stats_on_storage
                self._proxy.put_stats(host_id, xmlrpclib.Binary(data))
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in
            __call__
                return self.__send(self.__name, args)
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in
            __request
                verbose=self.__verbose
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in
            request
                return self.single_request(host, handler,
            request_body, verbose)
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1301, in
            single_request
                self.send_content(h, request_body)
              File "/usr/lib64/python2.7/xmlrpclib.py", line 1448, in
            send_content
                connection.endheaders(request_body)
              File "/usr/lib64/python2.7/httplib.py", line 1037, in
            endheaders
                self._send_output(message_body)
              File "/usr/lib64/python2.7/httplib.py", line 881, in
            _send_output
                self.send(msg)
              File "/usr/lib64/python2.7/httplib.py", line 843, in send
                self.connect()
              File
            
"/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/unixrpc.py",
            line 52, in connect
                self.sock.connect(base64.b16decode(self.host))
              File "/usr/lib64/python2.7/socket.py", line 224, in meth
                return getattr(self._sock,name)(*args)
            error: [Errno 2] No such file or directory

            MainThread::ERROR::2018-12-13
            
05:59:04,043::agent::145::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
            Trying to restart agent
            MainThread::INFO::2018-12-13
            
05:59:04,044::agent::89::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
            Agent shutting down
            MainThread::INFO::2018-12-13
            
05:59:14,923::agent::67::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
            ovirt-hosted-engine-ha agent 2.2.16 started



        --

        Cordialement,

        *Frank Soyer *
        Mob. 06 72 28 38 53 - Fix. 05 49 50 52 34

        <http://www.systea.net>

        *Systea IG*
        Administration systèmes, réseaux et bases de données
        www.systea.net <http://www.systea.net>
        Membre du réseau Les Professionnels du Numérique
        <http://www.spn.asso.fr>

        <http://www.kogite.fr>

        *KoGite*
        Hébergement de proximité
        www.kogite.fr <http://www.kogite.fr>



        Le Vendredi, Décembre 14, 2018 12:11 CET, Martin Sivak
        <msi...@redhat.com <mailto:msi...@redhat.com>> a écrit:
        Hi,

        no StartState is not common, it is only ever entered when the
        agent
        boots up. So something restarted or killed the agent process.
        Check
        the agent log in /var/log/ovirt-hosted-engine-ha for errors.

        Best regards

        Martin Sivak

        On Fri, Dec 14, 2018 at 12:05 PM fsoyer <fso...@systea.fr
        <mailto:fso...@systea.fr>> wrote:
        >
        > Hi Martin,
        > my problem is that nobody restarted the agent. Do you mean
        that this is not a normal behavior ? Is it possible that it
        restarts itself ?
        >
        > Thanks
        > --
        >
        > Regards,
        >
        > Frank
        >
        >
        >
        > Le Jeudi, Décembre 13, 2018 15:25 CET, Martin Sivak
        <msi...@redhat.com <mailto:msi...@redhat.com>> a écrit:
        >
        >
        > Hi,
        >
        > those are state change notifications from the hosted engine
        agent. It
        > basically means somebody restarted the ha-agent process and
        it found
        > out the VM is still running fine and returned to the proper
        state.
        >
        > Configuring it is possible using the broker.conf file in
        > /etc/ovirt-hosted-engine-ha (look for the notification
        section) or the
        > hosted-engine tool (search --help for set config) depending
        on the
        > version of hosted engine you are using.
        >
        > Best regards
        >
        > --
        > Martin Sivak
        >
        >
        > On Thu, Dec 13, 2018 at 3:10 PM fsoyer <fso...@systea.fr
        <mailto:fso...@systea.fr>> wrote:
        > >
        > > Hi,
        > > I don't find revelant answer about this. Sorry il this
        was already asked.
        > > I receive randomly (one or two tims a week, differents
        hours) 3 mails with this subjects :
        > > first : ovirt-hosted-engine state transition
        StartState-ReinitializeFSM
        > > second : ovirt-hosted-engine state transition
        ReinitializeFSM-EngineStarting
        > > third : ovirt-hosted-engine state transition
        EngineStarting-EngineUp
        > > all at exactly the same time. The "events" in GUI doesn't
        indicate anything about this. No impact on engine or VMs.
        > > So I wonder what this messages means ? And, if case of
        just "info" messages, is there a way to disable them ?
        > >
        > > Thanks.
        > > --
        > >
        > > Reagrds,
        > >
        > > Frank
        > >
        > > _______________________________________________
        > > Users mailing list -- users@ovirt.org
        <mailto:users@ovirt.org>
        > > To unsubscribe send an email to users-le...@ovirt.org
        <mailto:users-le...@ovirt.org>
        > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
        > > oVirt Code of Conduct:
        https://www.ovirt.org/community/about/community-guidelines/
        > > List Archives:
        
https://lists.ovirt.org/archives/list/users@ovirt.org/message/CVEHTWILWDEHASTCQHFHX62U4K4ZCOSK/
        >
        >
        >
        >



_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/5VJVLDA5J2CSYAYSIABZWF6Q6TOJIMI4/

_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: 
https://www.ovirt.org/community/about/community-guidelines/
List Archives: 
https://lists.ovirt.org/archives/list/users@ovirt.org/message/NJ263LV3D2TQH2NLAE4YV6Y3TATNX7KW/

Reply via email to