Re: [ovirt-users] VDSM service won't start

[Edward Haas]

On Fri, Jan 13, 2017 at 11:46 PM, paul.greene.va  wrote:

> Oh, I stumbled onto something relevant.
>
> I noticed on the host that was working correctly that the ifcfg-enp6s0
> file included a line for "BRIDGE=ovirtmgmt", and the other two didn't have
> that line. When I added that line to the other two hosts, and restarted
> networking, I was able to get those hosts in a status of "UP".
>
> That file is autogenerated by VDSM, so I wondered if it would survive a
> reboot. When I rebooted, the line had been removed again by VDSM.
>
> So, I guess the final question then is how to get persistence in keeping
> this BRIDGE line from getting removed across reboots?


VDSM on reboot will compare the current config to the persisted one and try
to sync it.
Perhaps you have a corrupted persistent configuration.

Could you please send us the following items:
- vdsm and supervdsm logs (from /var/log/vdsm)
- All your ifcfg files.
- The persisted VDSM network configuration from:
/var/lib/vdsm/persistence/netconf

Thanks,
Edy.


>
>
>
> On 1/13/2017 2:54 PM, Nir Soffer wrote:
>
>> On Fri, Jan 13, 2017 at 9:24 PM, paul.greene.va
>>  wrote:
>>
>>> Output below ...
>>>
>>>
>>>
>>> On 1/13/2017 1:47 PM, Nir Soffer wrote:
>>>
 On Fri, Jan 13, 2017 at 5:45 PM, paul.greene.va
  wrote:

> All,
>
> I'm having an issue with the vdsmd service refusing to start on a fresh
> install of RHEL 7.2, RHEV version 4.0.
>
> It initially came up correctly, and the command "ip a" showed a
> "vdsmdummy"
> interface and a "ovirtmgmt" interface. However after a couple of
> reboots,
> those interfaces disappeared, and running "systemctl status vdsmd"
> generated
> the message "Dependency failed for Virtual Desktop Service Manager/Job
> vdsmd.service/start failed with result 'dependency'". Didn't say what
> dependency though
>
> I have 3 hosts where this happening on 2 out of 3 hosts. For some odd
> reason, the one host isn't having any problems.
>
> In a Google search I found an instance where system clock timing was
> out
> of
> sync, and that messed it up. I checked all three hosts, as well as the
> RHEV
> manager and they all had chronyd running and the clocks appeared to be
> in
> sync.
>
> After a reboot the virtual interfaces usually initially come up, but go
> down
> again within a few minutes.
>
> Running journalctl -xe gives these three messages:
>
> "failed to start Virtual Desktop Server Manager network restoration"
>
> "Dependency failed for Virtual Desktop Server Manager"  (but it doesn't
> say
> which dependency failed"
>
> "Dependency failed for MOM instance configured for VDSM purposes"
> (again,
> doesn't way which dependency)
>
> Any suggestions?
>
 Can you share the output of:

 systemctl status vdsmd
 systemctl status mom
 systemctl status libvirtd
 journalctl -xe

 Nir

 Sure, here you go 
>>>
>>>
>>>
>>> [root@rhevh03 vdsm]# systemctl status vdsmd
>>> ā vdsmd.service - Virtual Desktop Server Manager
>>> Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled;
>>> vendor
>>> preset: enabled)
>>> Active: inactive (dead)
>>>
>>> Jan 13 12:01:53 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>>> Server Manager.
>>> Jan 13 12:01:53 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>>> result 'dependency'.
>>> Jan 13 13:51:50 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>>> Server Manager.
>>> Jan 13 13:51:50 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>>> result 'dependency'.
>>> Jan 13 13:55:15 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
>>> Server Manager.
>>> Jan 13 13:55:15 rhevh03 systemd[1]: Job vdsmd.service/start failed with
>>> result 'dependency'.
>>>
>>>
>>>
>>> [root@rhevh03 vdsm]# systemctl status momd
>>> ā momd.service - Memory Overcommitment Manager Daemon
>>> Loaded: loaded (/usr/lib/systemd/system/momd.service; static; vendor
>>> preset: disabled)
>>> Active: inactive (dead) since Fri 2017-01-13 13:53:09 EST; 2min 26s
>>> ago
>>>Process: 28294 ExecStart=/usr/sbin/momd -c /etc/momd.conf -d
>>> --pid-file
>>> /var/run/momd.pid (code=exited, status=0/SUCCESS)
>>>   Main PID: 28298 (code=exited, status=0/SUCCESS)
>>>
>>> Jan 13 13:53:09 rhevh03 systemd[1]: Starting Memory Overcommitment
>>> Manager
>>> Daemon...
>>> Jan 13 13:53:09 rhevh03 systemd[1]: momd.service: Supervising process
>>> 28298
>>> which is not our child. We'll most likely not notice when it exits.
>>> Jan 13 13:53:09 rhevh03 systemd[1]: Started Memory Overcommitment Manager
>>> Daemon.
>>> Jan 13 13:53:09 rhevh03 python[28298]: No worthy mechs found
>>>
>>>
>>>
>>> [root@rhevh03 vdsm]# systemctl status libvirtd
>>> ā libvirtd.service - Virtualization 

Re: [ovirt-users] VDSM service won't start

[paul.greene.va]

Oh, I stumbled onto something relevant.

I noticed on the host that was working correctly that the ifcfg-enp6s0 
file included a line for "BRIDGE=ovirtmgmt", and the other two didn't 
have that line. When I added that line to the other two hosts, and 
restarted networking, I was able to get those hosts in a status of "UP".


That file is autogenerated by VDSM, so I wondered if it would survive a 
reboot. When I rebooted, the line had been removed again by VDSM.


So, I guess the final question then is how to get persistence in keeping 
this BRIDGE line from getting removed across reboots?



On 1/13/2017 2:54 PM, Nir Soffer wrote:

On Fri, Jan 13, 2017 at 9:24 PM, paul.greene.va
 wrote:

Output below ...



On 1/13/2017 1:47 PM, Nir Soffer wrote:

On Fri, Jan 13, 2017 at 5:45 PM, paul.greene.va
 wrote:

All,

I'm having an issue with the vdsmd service refusing to start on a fresh
install of RHEL 7.2, RHEV version 4.0.

It initially came up correctly, and the command "ip a" showed a
"vdsmdummy"
interface and a "ovirtmgmt" interface. However after a couple of reboots,
those interfaces disappeared, and running "systemctl status vdsmd"
generated
the message "Dependency failed for Virtual Desktop Service Manager/Job
vdsmd.service/start failed with result 'dependency'". Didn't say what
dependency though

I have 3 hosts where this happening on 2 out of 3 hosts. For some odd
reason, the one host isn't having any problems.

In a Google search I found an instance where system clock timing was out
of
sync, and that messed it up. I checked all three hosts, as well as the
RHEV
manager and they all had chronyd running and the clocks appeared to be in
sync.

After a reboot the virtual interfaces usually initially come up, but go
down
again within a few minutes.

Running journalctl -xe gives these three messages:

"failed to start Virtual Desktop Server Manager network restoration"

"Dependency failed for Virtual Desktop Server Manager"  (but it doesn't
say
which dependency failed"

"Dependency failed for MOM instance configured for VDSM purposes"
(again,
doesn't way which dependency)

Any suggestions?

Can you share the output of:

systemctl status vdsmd
systemctl status mom
systemctl status libvirtd
journalctl -xe

Nir


Sure, here you go 



[root@rhevh03 vdsm]# systemctl status vdsmd
ā vdsmd.service - Virtual Desktop Server Manager
Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor
preset: enabled)
Active: inactive (dead)

Jan 13 12:01:53 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
Server Manager.
Jan 13 12:01:53 rhevh03 systemd[1]: Job vdsmd.service/start failed with
result 'dependency'.
Jan 13 13:51:50 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
Server Manager.
Jan 13 13:51:50 rhevh03 systemd[1]: Job vdsmd.service/start failed with
result 'dependency'.
Jan 13 13:55:15 rhevh03 systemd[1]: Dependency failed for Virtual Desktop
Server Manager.
Jan 13 13:55:15 rhevh03 systemd[1]: Job vdsmd.service/start failed with
result 'dependency'.



[root@rhevh03 vdsm]# systemctl status momd
ā momd.service - Memory Overcommitment Manager Daemon
Loaded: loaded (/usr/lib/systemd/system/momd.service; static; vendor
preset: disabled)
Active: inactive (dead) since Fri 2017-01-13 13:53:09 EST; 2min 26s ago
   Process: 28294 ExecStart=/usr/sbin/momd -c /etc/momd.conf -d --pid-file
/var/run/momd.pid (code=exited, status=0/SUCCESS)
  Main PID: 28298 (code=exited, status=0/SUCCESS)

Jan 13 13:53:09 rhevh03 systemd[1]: Starting Memory Overcommitment Manager
Daemon...
Jan 13 13:53:09 rhevh03 systemd[1]: momd.service: Supervising process 28298
which is not our child. We'll most likely not notice when it exits.
Jan 13 13:53:09 rhevh03 systemd[1]: Started Memory Overcommitment Manager
Daemon.
Jan 13 13:53:09 rhevh03 python[28298]: No worthy mechs found



[root@rhevh03 vdsm]# systemctl status libvirtd
ā libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor
preset: enabled)
   Drop-In: /etc/systemd/system/libvirtd.service.d
āāunlimited-core.conf
Active: active (running) since Fri 2017-01-13 13:50:47 EST; 8min ago
  Docs: man:libvirtd(8)
http://libvirt.org
  Main PID: 27964 (libvirtd)
CGroup: /system.slice/libvirtd.service
āā27964 /usr/sbin/libvirtd --listen

Jan 13 13:50:47 rhevh03 systemd[1]: Starting Virtualization daemon...
Jan 13 13:50:47 rhevh03 systemd[1]: Started Virtualization daemon.
Jan 13 13:53:09 rhevh03 libvirtd[27964]: libvirt version: 2.0.0, package:
10.el7_3.2 (Red Hat, Inc. ,
2016-11-10-04:43:57, x86-034.build.eng.bos.redhat.com)
Jan 13 13:53:09 rhevh03 libvirtd[27964]: hostname: rhevh03
Jan 13 13:53:09 rhevh03 libvirtd[27964]: End of file while reading data:
Input/output error


[root@rhevh03 vdsm]# journalctl -xe
Jan 13 13:55:15 rhevh03 vdsm-tool[28334]: 

Re: [ovirt-users] VDSM service won't start

[Dominik Holler]

On Fri, 13 Jan 2017 10:45:43 -0500
"paul.greene.va"  wrote:

> 
> After a reboot the virtual interfaces usually initially come up, but
> go down again within a few minutes.
> 
> Running journalctl -xe gives these three messages:
> 
> "failed to start Virtual Desktop Server Manager network restoration"
> 
> "Dependency failed for Virtual Desktop Server Manager"  (but it
> doesn't say which dependency failed"
> 
> "Dependency failed for MOM instance configured for VDSM purposes"  
> (again, doesn't way which dependency)
> 
> Any suggestions?
> 

Is libvirtd.service running?
___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] VDSM service won't start

[paul.greene.va]

All,

I'm having an issue with the vdsmd service refusing to start on a fresh 
install of RHEL 7.2, RHEV version 4.0.


It initially came up correctly, and the command "ip a" showed a 
"vdsmdummy" interface and a "ovirtmgmt" interface. However after a 
couple of reboots, those interfaces disappeared, and running "systemctl 
status vdsmd" generated the message "Dependency failed for Virtual 
Desktop Service Manager/Job vdsmd.service/start failed with result 
'dependency'". Didn't say what dependency though


I have 3 hosts where this happening on 2 out of 3 hosts. For some odd 
reason, the one host isn't having any problems.


In a Google search I found an instance where system clock timing was out 
of sync, and that messed it up. I checked all three hosts, as well as 
the RHEV manager and they all had chronyd running and the clocks 
appeared to be in sync.


After a reboot the virtual interfaces usually initially come up, but go 
down again within a few minutes.


Running journalctl -xe gives these three messages:

"failed to start Virtual Desktop Server Manager network restoration"

"Dependency failed for Virtual Desktop Server Manager"  (but it doesn't 
say which dependency failed"


"Dependency failed for MOM instance configured for VDSM purposes"  
(again, doesn't way which dependency)


Any suggestions?

Paul

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

[ovirt-users] VDSM will not start

[Jonas Kirk Pedersen]

Had a problem where on the host the vdsm service would not start. The 
following is from #ovirt IRC chat after I emailed the sos report from 
the host


13:29:06) sbonazzo: jkp looks like an issue while starting supervdsm, I 
see:
(13:29:08) sbonazzo: Mar 9 12:41:30 kvm1 daemonAdapter: OSError: [Errno 
2] No such file or directory: '/var/run/vdsm/svdsm.sock'
(13:29:08) sbonazzo: Mar 9 12:41:30 kvm1 systemd: start request repeated 
too quickly for supervdsmd.service

(13:29:26) sbonazzo: danken: ybronhei: nsoffer ^^
(13:30:57) sbonazzo: msivak: i see also Mar 9 12:27:18 kvm1 systemd: 
[/usr/lib/systemd/system/momd.service:11] Unknown lvalue 'Timeout' in 
section 'Service'

(13:31:30) sbonazzo: msivak: maybe epel7 mom should be updated
(13:32:33) sbonazzo: after that, vdsm doesn't come up due to Mar 9 
12:27:55 kvm1 daemonAdapter: OSError: [Errno 2] No such file or 
directory: '/var/run/vdsm/svdsm.sock since supervdsmd didn't run

(13:34:50) sbonazzo: jkp: can you try to manually start supervdsmd service?
(13:35:07) jkp: sbonazzo, it is confirmed that the file is not there. 
But shouldn't the service create this file -- Yes I'll try

(13:35:19) sbonazzo: jkp: yes, it's supposed to
(13:39:07) jkp: sbonazzo, still gives me the : OSError: [Errno 2] No 
such file or directory: '/var/run/vdsm/svdsm.sock'

(13:39:35) sbonazzo: jkp does /var/run/vdsm/ exist?
(13:40:55) jkp: sbonazzo, yes it contains [root@kvm1 ~]# ls 
/var/run/vdsm/ lvm sourceRoutes trackedInterfaces v2v


Later I received a email

Mar  9 10:41:21 kvm1 journal: Module 
/usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not accessible
Mar  9 10:41:21 kvm1 journal: The server certificate 
/etc/pki/vdsm/certs/vdsmcert.pem is not yet active
Mar  9 10:41:21 kvm1 systemd: libvirtd.service: main process exited, 
code=exited, status=6/NOTCONFIGURED
Mar  9 10:41:21 kvm1 systemd: Failed to start Virtualization daemon.
Mar  9 10:41:21 kvm1 systemd: Dependency failed for Virtual Desktop Server 
Manager.
Mar  9 10:41:21 kvm1 systemd: Dependency failed for MOM instance configured for 
VDSM purposes.
Mar  9 10:41:21 kvm1 systemd: Job mom-vdsm.service/start failed with result 
'dependency'.
Mar  9 10:41:21 kvm1 systemd: Job vdsmd.service/start failed with result 
'dependency'.
Mar  9 10:41:21 kvm1 systemd: Dependency failed for Virtual Desktop Server 
Manager network restoration.
Mar  9 10:41:21 kvm1 systemd: Job vdsm-network.service/start failed with result 
'dependency'.
Mar  9 10:41:21 kvm1 systemd: Unit libvirtd.service entered failed state.
Mar  9 10:41:21 kvm1 systemd: libvirtd.service failed.
Mar  9 10:41:21 kvm1 systemd-logind: Removed session 4.
Mar  9 10:41:21 kvm1 systemd: libvirtd.service holdoff time over, scheduling 
restart.

I see that libvirtd fails to start.
It might be due to Engine system clock being deep in the future?

What I had failed to notice was that the vdsmcert.pem that was not active.
The ntp service running on the Engine was out of sync with a year 
resulting in that of cause the certificate did not work and then the 
vdsm would not start.
I have a fairly new engine so I removed it and fixed the time issue and 
all was perfect


Thanks for a great help on the chat and on email

--
Jonas Kirk Pedersen
ASOM-Net
Systemadministrator

___
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users