Re: [ovirt-users] 4.0 : hosts connecting/non responsive and data domains inactive

2017-05-11 Thread Piotr Kliczewski 
>From provided log snippets I see that engine was unable to connect to both
of your hosts. Is the network OK?

In vdsm log it seems that there is an issue with you storage domain.

I am not able to say anything about the issue you are facing without having
more complete logs.

On Thu, May 11, 2017 at 6:28 PM, Oved Ourfali  wrote:

> What 4.0 version do you use?
> Can you attach complete engine and vdsm logs?
> CC-ing Piotr as well.
>
>
> On May 11, 2017 6:44 PM, "Alexis HAUSER" 
> wrote:
>
>> After rebooting the manager VM,  hosts are connecting/non responsive and
>> data domains inactive. Here are the engine and vdsmd logs. Any ideas ?
>>
>>
>>
>>
>> Engine logs :
>>
>>
>>
>> 2017-05-11 17:28:09,302 WARN  [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5)
>> [55f1aab5] Correlation ID: null, Call Stack: null, Custom Event ID: -1,
>> Message: Failed to verify Power Management configuration for Host
>> rhvserv-05.
>> 2017-05-11 17:28:09,346 INFO  
>> [org.ovirt.engine.core.bll.HandleVdsVersionCommand]
>> (DefaultQuartzScheduler5) [48bc69cd] Running command:
>> HandleVdsVersionCommand internal: true. Entities affected :  ID:
>> 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS
>> 2017-05-11 17:28:09,349 WARN  [org.ovirt.engine.core.vdsbroker.VdsManager]
>> (org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not
>> responding.
>> 2017-05-11 17:28:09,364 WARN  [org.ovirt.engine.core.dal.dbb
>> roker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-27)
>> [639977e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1,
>> Message: Host rhvserv-05 is not responding. Host cannot be fenced
>> automatically because power management for the host is disabled.
>> 2017-05-11 17:28:11,299 ERROR [org.ovirt.engine.core.vdsbrok
>> er.vdsbroker.GetCapabilitiesVDSCommand] (DefaultQuartzScheduler3)
>> [c0e6a2e] Command 'GetCapabilitiesVDSCommand(HostName = rhvserv-03,
>> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
>> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916',
>> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution
>> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>> Connection failed
>> 2017-05-11 17:28:11,299 ERROR 
>> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
>> (DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03'
>> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
>> Connection failed
>> 2017-05-11 17:28:11,327 INFO  
>> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient]
>> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.16
>> 8.93.214
>> 2017-05-11 17:28:12,484 INFO  [org.ovirt.engine.core.vdsbrok
>> er.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3)
>> [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName = rhvserv-05,
>> VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
>> hostId='04565f10-9abf-4709-9445-9dc6ed97e136',
>> vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id:
>> f807ece
>> 2017-05-11 17:28:12,487 INFO  [org.ovirt.engine.core.vdsbrok
>> er.vdsbroker.GetHardwareInfoVDSCommand] (DefaultQuartzScheduler3)
>> [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log id: f807ece
>> 2017-05-11 17:28:12,532 INFO  [org.ovirt.engine.core.bll.Han
>> dleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler3)
>> [4e882ea0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand
>> internal: true. Entities affected :  ID: 04565f10-9abf-4709-9445-9dc6ed97e136
>> Type: VDS
>> 2017-05-11 17:28:12,539 INFO  [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
>> (DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand
>> internal: true. Entities affected :  ID: 58f8df36-019f-02bc-00e7-0023
>> Type: StoragePool
>> 2017-05-11 17:28:12,545 INFO  [org.ovirt.engine.core.bll.sto
>> rage.pool.ConnectHostToStoragePoolServersCommand]
>> (DefaultQuartzScheduler3) [46cc3f58] Running command:
>> ConnectHostToStoragePoolServersCommand internal: true. Entities affected
>> :  ID: 58f8df36-019f-02bc-00e7-0023 Type: StoragePool
>> 2017-05-11 17:28:12,556 INFO  [org.ovirt.engine.core.vdsbrok
>> er.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3)
>> [46cc3f58] START, ConnectStorageServerVDSCommand(HostName = rhvserv-05,
>> StorageServerConnectionManagementVDSParameters:{runAsync='true',
>> hostId='04565f10-9abf-4709-9445-9dc6ed97e136',
>> storagePoolId='58f8df36-019f-02bc-00e7-0023',
>> storageType='ISCSI', connectionList='[StorageServer
>> Connections:{id='10c0528b-f08d-4d1d-8c63-8a05fd9d58b9',
>> connection='10.35.21.1', iqn='iqn.1984-05.com.dell:powe
>> rvault.md3200i.6782bcb00073e3324edde164', vfsType='null',
>> mountOptions='null', nfsVersion='null', nfsRetrans='null', nfsTimeo='null',
>> iface='null', netIfaceName='null'}]'}), log id: 1beb27b6
>> 2017-05-11 17:28:13,031

Re: [ovirt-users] 4.0 : hosts connecting/non responsive and data domains inactive

2017-05-11 Thread Oved Ourfali 
What 4.0 version do you use?
Can you attach complete engine and vdsm logs?
CC-ing Piotr as well.


On May 11, 2017 6:44 PM, "Alexis HAUSER" 
wrote:

> After rebooting the manager VM,  hosts are connecting/non responsive and
> data domains inactive. Here are the engine and vdsmd logs. Any ideas ?
>
>
>
>
> Engine logs :
>
>
>
> 2017-05-11 17:28:09,302 WARN  [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler5)
> [55f1aab5] Correlation ID: null, Call Stack: null, Custom Event ID: -1,
> Message: Failed to verify Power Management configuration for Host
> rhvserv-05.
> 2017-05-11 17:28:09,346 INFO  
> [org.ovirt.engine.core.bll.HandleVdsVersionCommand]
> (DefaultQuartzScheduler5) [48bc69cd] Running command:
> HandleVdsVersionCommand internal: true. Entities affected :  ID:
> 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS
> 2017-05-11 17:28:09,349 WARN  [org.ovirt.engine.core.vdsbroker.VdsManager]
> (org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not
> responding.
> 2017-05-11 17:28:09,364 WARN  [org.ovirt.engine.core.dal.
> dbbroker.auditloghandling.AuditLogDirector] 
> (org.ovirt.thread.pool-6-thread-27)
> [639977e4] Correlation ID: null, Call Stack: null, Custom Event ID: -1,
> Message: Host rhvserv-05 is not responding. Host cannot be fenced
> automatically because power management for the host is disabled.
> 2017-05-11 17:28:11,299 ERROR 
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand]
> (DefaultQuartzScheduler3) [c0e6a2e] Command 
> 'GetCapabilitiesVDSCommand(HostName
> = rhvserv-03, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916',
> vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution
> failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
> Connection failed
> 2017-05-11 17:28:11,299 ERROR 
> [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring]
> (DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03'
> runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException:
> Connection failed
> 2017-05-11 17:28:11,327 INFO  
> [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient]
> (SSL Stomp Reactor) [] Connecting to rhvserv-04.mydomain.com/192.
> 168.93.214
> 2017-05-11 17:28:12,484 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> (DefaultQuartzScheduler3) [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName
> = rhvserv-05, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
> hostId='04565f10-9abf-4709-9445-9dc6ed97e136',
> vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id:
> f807ece
> 2017-05-11 17:28:12,487 INFO  
> [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand]
> (DefaultQuartzScheduler3) [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log
> id: f807ece
> 2017-05-11 17:28:12,532 INFO  [org.ovirt.engine.core.bll.
> HandleVdsCpuFlagsOrClusterChangedCommand] (DefaultQuartzScheduler3)
> [4e882ea0] Running command: HandleVdsCpuFlagsOrClusterChangedCommand
> internal: true. Entities affected :  ID: 04565f10-9abf-4709-9445-9dc6ed97e136
> Type: VDS
> 2017-05-11 17:28:12,539 INFO  [org.ovirt.engine.core.bll.InitVdsOnUpCommand]
> (DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand
> internal: true. Entities affected :  ID: 58f8df36-019f-02bc-00e7-0023
> Type: StoragePool
> 2017-05-11 17:28:12,545 INFO  [org.ovirt.engine.core.bll.storage.pool.
> ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler3)
> [46cc3f58] Running command: ConnectHostToStoragePoolServersCommand
> internal: true. Entities affected :  ID: 58f8df36-019f-02bc-00e7-0023
> Type: StoragePool
> 2017-05-11 17:28:12,556 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.
> ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58]
> START, ConnectStorageServerVDSCommand(HostName = rhvserv-05,
> StorageServerConnectionManagementVDSParameters:{runAsync='true',
> hostId='04565f10-9abf-4709-9445-9dc6ed97e136',
> storagePoolId='58f8df36-019f-02bc-00e7-0023',
> storageType='ISCSI', connectionList='[StorageServerConnections:{id='
> 10c0528b-f08d-4d1d-8c63-8a05fd9d58b9', connection='10.35.21.1',
> iqn='iqn.1984-05.com.dell:powervault.md3200i.
> 6782bcb00073e3324edde164', vfsType='null', mountOptions='null',
> nfsVersion='null', nfsRetrans='null', nfsTimeo='null', iface='null',
> netIfaceName='null'}]'}), log id: 1beb27b6
> 2017-05-11 17:28:13,031 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.
> ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58]
> FINISH, ConnectStorageServerVDSCommand, return: 
> {10c0528b-f08d-4d1d-8c63-8a05fd9d58b9=0},
> log id: 1beb27b6
> 2017-05-11 17:28:13,032 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.
> ConnectStorageServerVDSCommand] (DefaultQuartzScheduler3) [46cc3f58]
> START, ConnectStorageServerVDSCommand(HostName = rhvserv-05,

[ovirt-users] 4.0 : hosts connecting/non responsive and data domains inactive

2017-05-11 Thread Alexis HAUSER 
After rebooting the manager VM, hosts are connecting/non responsive and data 
domains inactive. Here are the engine and vdsmd logs. Any ideas ? 




Engine logs : 



2017-05-11 17:28:09,302 WARN 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(DefaultQuartzScheduler5) [55f1aab5] Correlation ID: null, Call Stack: null, 
Custom Event ID: -1, Message: Failed to verify Power Management configuration 
for Host rhvserv-05. 
2017-05-11 17:28:09,346 INFO 
[org.ovirt.engine.core.bll.HandleVdsVersionCommand] (DefaultQuartzScheduler5) 
[48bc69cd] Running command: HandleVdsVersionCommand internal: true. Entities 
affected : ID: 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS 
2017-05-11 17:28:09,349 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] 
(org.ovirt.thread.pool-6-thread-27) [639977e4] Host 'rhvserv-05' is not 
responding. 
2017-05-11 17:28:09,364 WARN 
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] 
(org.ovirt.thread.pool-6-thread-27) [639977e4] Correlation ID: null, Call 
Stack: null, Custom Event ID: -1, Message: Host rhvserv-05 is not responding. 
Host cannot be fenced automatically because power management for the host is 
disabled. 
2017-05-11 17:28:11,299 ERROR 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesVDSCommand] 
(DefaultQuartzScheduler3) [c0e6a2e] Command 'GetCapabilitiesVDSCommand(HostName 
= rhvserv-03, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', 
hostId='4036f027-8e90-49c0-8ca5-3ddb8d586916', 
vds='Host[rhvserv-03,4036f027-8e90-49c0-8ca5-3ddb8d586916]'})' execution 
failed: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: Connection 
failed 
2017-05-11 17:28:11,299 ERROR 
[org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] 
(DefaultQuartzScheduler3) [c0e6a2e] Failure to refresh host 'rhvserv-03' 
runtime info: org.ovirt.vdsm.jsonrpc.client.ClientConnectionException: 
Connection failed 
2017-05-11 17:28:11,327 INFO 
[org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] 
Connecting to rhvserv-04.mydomain.com/192.168.93.214 
2017-05-11 17:28:12,484 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] 
(DefaultQuartzScheduler3) [c0e6a2e] START, GetHardwareInfoVDSCommand(HostName = 
rhvserv-05, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true', 
hostId='04565f10-9abf-4709-9445-9dc6ed97e136', 
vds='Host[rhvserv-05,04565f10-9abf-4709-9445-9dc6ed97e136]'}), log id: f807ece 
2017-05-11 17:28:12,487 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoVDSCommand] 
(DefaultQuartzScheduler3) [c0e6a2e] FINISH, GetHardwareInfoVDSCommand, log id: 
f807ece 
2017-05-11 17:28:12,532 INFO 
[org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] 
(DefaultQuartzScheduler3) [4e882ea0] Running command: 
HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : 
ID: 04565f10-9abf-4709-9445-9dc6ed97e136 Type: VDS 
2017-05-11 17:28:12,539 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] 
(DefaultQuartzScheduler3) [75f25b35] Running command: InitVdsOnUpCommand 
internal: true. Entities affected : ID: 58f8df36-019f-02bc-00e7-0023 
Type: StoragePool 
2017-05-11 17:28:12,545 INFO 
[org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] 
(DefaultQuartzScheduler3) [46cc3f58] Running command: 
ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 
58f8df36-019f-02bc-00e7-0023 Type: StoragePool 
2017-05-11 17:28:12,556 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] 
(DefaultQuartzScheduler3) [46cc3f58] START, 
ConnectStorageServerVDSCommand(HostName = rhvserv-05, 
StorageServerConnectionManagementVDSParameters:{runAsync='true', 
hostId='04565f10-9abf-4709-9445-9dc6ed97e136', 
storagePoolId='58f8df36-019f-02bc-00e7-0023', storageType='ISCSI', 
connectionList='[StorageServerConnections:{id='10c0528b-f08d-4d1d-8c63-8a05fd9d58b9',
 connection='10.35.21.1', 
iqn='iqn.1984-05.com.dell:powervault.md3200i.6782bcb00073e3324edde164', 
vfsType='null', mountOptions='null', nfsVersion='null', nfsRetrans='null', 
nfsTimeo='null', iface='null', netIfaceName='null'}]'}), log id: 1beb27b6 
2017-05-11 17:28:13,031 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] 
(DefaultQuartzScheduler3) [46cc3f58] FINISH, ConnectStorageServerVDSCommand, 
return: {10c0528b-f08d-4d1d-8c63-8a05fd9d58b9=0}, log id: 1beb27b6 
2017-05-11 17:28:13,032 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] 
(DefaultQuartzScheduler3) [46cc3f58] START, 
ConnectStorageServerVDSCommand(HostName = rhvserv-05, 
StorageServerConnectionManagementVDSParameters:{runAsync='true', 
hostId='04565f10-9abf-4709-9445-9dc6ed97e136', 
storagePoolId='58f8df36-019f-02bc-00e7-0023', storageType='NFS', 
connectionList='[StorageServerConnections:{id='e604d0d2-0810-4c25-b9ed-610f9923cb1a',
 connection='nfsserv-01:/nfs/export',