[ovirt-users] Re: OVirt Gluster Fail
Hi Andrea, The cluster volumes might have sharding enabled and thus files larger than shard size can be recovered only via cluster. You can try to restart gluster on all nodes and force heal: 1. Kill gluster processes: systemctl stop glusterd /usr/share/glusterfs/scripts/stop-all-gluster-processes.sh 2. Start gluster: systemctl start glusterd 3. Force heal: for i in $(gluster volume list); do gluster volume heal $i full ; done sleep 300 for i in $(gluster volume list); do gluster volume heal $i info summary ; done Best Regards, Strahil NikolovOn Mar 23, 2019 13:51, commram...@tiscali.it wrote: > > During maintenance of a machine the hosted engine crashed. > At that point there was no more chance of managing anything. > > The VMs have paused, and were no longer manageable. > I restarted the machine, but one point all the bricks were no longer reachable. > > Now I am in a situation where the engine support is no longer loaded. > > The gluster sees the peers connected and the services turned on for the various bricks, but fails to heal the messages that I find for each machine are the following > > # gluster volume heal engine info > Brick 192.170.254.3:/bricks/engine/brick > > . > . > . > > Status: Connected Number of entries: 190 > > Brick 192.170.254.4:/bricks/engine/brick > Status: Il socket di destinazione non è connesso > Number of entries: - > > Brick 192.170.254.6:/bricks/engine/brick > Status: Il socket di destinazione non è connesso > Number of entries: - > > this for all the bricks (some have no heal to do because the machines inside were turned off). > > In practice all the bricks see only localhost as connected. > > How can I restore the machines? > Is there a way to read data from the physical machine and export it so that it can be reused? > Unfortunately we need to access that data. > > Someone can help me. > > Thanks Andrea > ___ > Users mailing list -- users@ovirt.org > To unsubscribe send an email to users-le...@ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/EOIY7ZU4GOEMRUNY3CWF6R3JIQNPHLVA/___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NMBDYBOY4TZB37I6O6VYBCVVGM5H3Y3F/
[ovirt-users] Re: How to fix ovn apparent inconsistency?
On Sat, 23 Mar 2019 15:19:06 +0100
Gianluca Cecchi wrote:
> On Fri, Mar 22, 2019 at 10:19 PM Dominik Holler wrote:
>
> >
> > > in _runHooksDir
> > > raise exception.HookError(err)
> > > HookError: Hook Error: ('',)
> >
> > Thanks for raising this.
> > I created https://bugzilla.redhat.com/1691933 to track this.
> >
> > Do you uninstalled vdsm-hook-openstacknet?
> >
>
> No.
> It seems to me this package had never got installed but in 4.2.x OVN
> external network provider worked.
> The environment was created at beginning 2017 with 4.0.6 and then gradually
> updated, now at 4.3.2.
> OVN originally installed when in 4.1.0 with the manual way before official
> inclusion in engine-setup
>
> [root@ov300 ~]# rpm -q vdsm-hook-openstacknet
> package vdsm-hook-openstacknet is not installed
> [root@ov300 ~]#
>
> [root@ov300 ~]# ll -rt /var/log/yum.log*
> -rw---. 1 root root 63893 Sep 29 2017 /var/log/yum.log-20180101
> -rw---. 1 root root 13840 Feb 9 2018 /var/log/yum.log-20180326
> -rw---. 1 root root 43106 Nov 22 11:47 /var/log/yum.log-20190101
> -rw---. 1 root root 38473 Mar 5 13:46 /var/log/yum.log-20190306
> -rw---. 1 root root 5018 Mar 22 14:11 /var/log/yum.log
> [root@ov300 ~]#
>
> [root@ov300 ~]# grep vdsm-hook-openstacknet /var/log/yum.log*
> [root@ov300 ~]#
>
> And the same for the other two hosts
> I can confirm that if I install that package (no vdsm restart):
>
> Installing:
> vdsm-hook-openstacknet noarch
> 4.30.11-1.el7 ovirt-4.3 14 k
>
> The VM with OVN network card on ovn192 is able to boot now and I have the
> vnet1 interface on ov300
>
> [root@ov300 ~]# ovs-vsctl show
> f1a41e9c-16fb-4aa2-a386-2f366ade4d3c
> Bridge br-int
> fail_mode: secure
> Port br-int
> Interface br-int
> type: internal
> Port "ovn-b8872a-0"
> Interface "ovn-b8872a-0"
> type: geneve
> options: {csum="true", key=flow, remote_ip="10.4.192.34"}
> Port "ovn-1dce5b-0"
> Interface "ovn-1dce5b-0"
> type: geneve
> options: {csum="true", key=flow, remote_ip="10.4.192.32"}
> Port "vnet1"
> Interface "vnet1"
> ovs_version: "2.10.1"
> [root@ov300 ~]#
>
> [root@ovmgr1 ~]# ovn-sbctl show
> Chassis "ddecf0da-4708-4f93-958b-6af365a5eeca"
> hostname: "ov300.datacenter.polimi.it"
> Encap geneve
> ip: "10.4.192.33"
> options: {csum="true"}
> Port_Binding "84c78095-744c-4415-805f-5f739af3d4d3"
> Chassis "1dce5b7c-a9fc-4ddb-99b4-e2c9e0fa54c5"
> hostname: "ov200.datacenter.polimi.it"
> Encap geneve
> ip: "10.4.192.32"
> options: {csum="true"}
> Chassis "b8872ab5-4606-4a79-b77d-9d956a18d349"
> hostname: "ov301.datacenter.polimi.it"
> Encap geneve
> ip: "10.4.192.34"
> options: {csum="true"}
> [root@ovmgr1 ~]#
>
> And on engine:
> [root@ovmgr1 ~]# ovn-nbctl show
> switch fc2fc4e8-ff71-4ec3-ba03-536a870cd483
> (ovirt-ovn192-1e252228-ade7-47c8-acda-5209be358fcf)
> port 84c78095-744c-4415-805f-5f739af3d4d3
> addresses: ["00:1a:4a:17:01:53 dynamic"]
> switch 9e77163a-c4e4-4abf-a554-0388e6b5e4ce
> (ovirt-ovn172-4ac7ba24-aad5-432d-b1d2-672eaeea7d63)
> [root@ovmgr1 ~]#
>
> So at the end it could be a missing dependency during install of new
> packages?
>
Not by intention. If vdsm-hook-openstacknet is installed, a file in
/etc/sudoers.d/ is created, which allows vdsm to call ovs-vsctl without
restricted parameters.
/etc/sudoers.d/50_vdsm_hook_ovirt_provider_ovn_hook of ovirt-provider-ovn-driver
should allow vdsm to call ovs-vsctl with all required parameters, but
it does not. This is why I created bug 1691933.
In the newer installations I checked vdsm-hook-openstacknet was
installed and hides the bug.
Maybe there are upgrade paths, which results in scenarios, where
vdsm-hook-openstacknet is not installed, which should be fine, but shows
the bug.
> I have to dig a bit more, because from first tests if I start another VM on
> the same ovn192 network also on the same host they are not able to
> communicate
> Possibly an iptables misconfiguration on host?
>
Just to understand the error, would you please check if
/var/log/openvswitch/ovn-controller.log
or any other logfile in the same directory contains any hints?
Would communication using a new created ovn network without port
security enabled work?
If there are not further hints, I suggest to re-configure the
ovirt-provider-ovn-driver on the host via
vdsm-tool ovn-config OVN_Central_IP Tunneling_IP_or_Network_Name
(please find more details on
https://ovirt.org/documentation/admin-guide/chap-External_Providers.html#configuring-hosts-for-an-ovn-tunnel-networ
)
and check if this fixed the issue.
> I have vnet1 and vnet2 on host now
>
> [root@ov300 ~]# ovs-vsctl show
> f1a41e9c-16fb-4aa2-a386-2f366ade4d3c
> Bridge br-int
> fail_mode: secure
>
[ovirt-users] Re: How to fix ovn apparent inconsistency?
On Fri, Mar 22, 2019 at 10:19 PM Dominik Holler wrote:
>
> > in _runHooksDir
> > raise exception.HookError(err)
> > HookError: Hook Error: ('',)
>
> Thanks for raising this.
> I created https://bugzilla.redhat.com/1691933 to track this.
>
> Do you uninstalled vdsm-hook-openstacknet?
>
No.
It seems to me this package had never got installed but in 4.2.x OVN
external network provider worked.
The environment was created at beginning 2017 with 4.0.6 and then gradually
updated, now at 4.3.2.
OVN originally installed when in 4.1.0 with the manual way before official
inclusion in engine-setup
[root@ov300 ~]# rpm -q vdsm-hook-openstacknet
package vdsm-hook-openstacknet is not installed
[root@ov300 ~]#
[root@ov300 ~]# ll -rt /var/log/yum.log*
-rw---. 1 root root 63893 Sep 29 2017 /var/log/yum.log-20180101
-rw---. 1 root root 13840 Feb 9 2018 /var/log/yum.log-20180326
-rw---. 1 root root 43106 Nov 22 11:47 /var/log/yum.log-20190101
-rw---. 1 root root 38473 Mar 5 13:46 /var/log/yum.log-20190306
-rw---. 1 root root 5018 Mar 22 14:11 /var/log/yum.log
[root@ov300 ~]#
[root@ov300 ~]# grep vdsm-hook-openstacknet /var/log/yum.log*
[root@ov300 ~]#
And the same for the other two hosts
I can confirm that if I install that package (no vdsm restart):
Installing:
vdsm-hook-openstacknet noarch
4.30.11-1.el7 ovirt-4.3 14 k
The VM with OVN network card on ovn192 is able to boot now and I have the
vnet1 interface on ov300
[root@ov300 ~]# ovs-vsctl show
f1a41e9c-16fb-4aa2-a386-2f366ade4d3c
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "ovn-b8872a-0"
Interface "ovn-b8872a-0"
type: geneve
options: {csum="true", key=flow, remote_ip="10.4.192.34"}
Port "ovn-1dce5b-0"
Interface "ovn-1dce5b-0"
type: geneve
options: {csum="true", key=flow, remote_ip="10.4.192.32"}
Port "vnet1"
Interface "vnet1"
ovs_version: "2.10.1"
[root@ov300 ~]#
[root@ovmgr1 ~]# ovn-sbctl show
Chassis "ddecf0da-4708-4f93-958b-6af365a5eeca"
hostname: "ov300.datacenter.polimi.it"
Encap geneve
ip: "10.4.192.33"
options: {csum="true"}
Port_Binding "84c78095-744c-4415-805f-5f739af3d4d3"
Chassis "1dce5b7c-a9fc-4ddb-99b4-e2c9e0fa54c5"
hostname: "ov200.datacenter.polimi.it"
Encap geneve
ip: "10.4.192.32"
options: {csum="true"}
Chassis "b8872ab5-4606-4a79-b77d-9d956a18d349"
hostname: "ov301.datacenter.polimi.it"
Encap geneve
ip: "10.4.192.34"
options: {csum="true"}
[root@ovmgr1 ~]#
And on engine:
[root@ovmgr1 ~]# ovn-nbctl show
switch fc2fc4e8-ff71-4ec3-ba03-536a870cd483
(ovirt-ovn192-1e252228-ade7-47c8-acda-5209be358fcf)
port 84c78095-744c-4415-805f-5f739af3d4d3
addresses: ["00:1a:4a:17:01:53 dynamic"]
switch 9e77163a-c4e4-4abf-a554-0388e6b5e4ce
(ovirt-ovn172-4ac7ba24-aad5-432d-b1d2-672eaeea7d63)
[root@ovmgr1 ~]#
So at the end it could be a missing dependency during install of new
packages?
I have to dig a bit more, because from first tests if I start another VM on
the same ovn192 network also on the same host they are not able to
communicate
Possibly an iptables misconfiguration on host?
I have vnet1 and vnet2 on host now
[root@ov300 ~]# ovs-vsctl show
f1a41e9c-16fb-4aa2-a386-2f366ade4d3c
Bridge br-int
fail_mode: secure
Port br-int
Interface br-int
type: internal
Port "vnet2"
Interface "vnet2"
Port "ovn-b8872a-0"
Interface "ovn-b8872a-0"
type: geneve
options: {csum="true", key=flow, remote_ip="10.4.192.34"}
Port "ovn-1dce5b-0"
Interface "ovn-1dce5b-0"
type: geneve
options: {csum="true", key=flow, remote_ip="10.4.192.32"}
Port "vnet1"
Interface "vnet1"
ovs_version: "2.10.1"
[root@ov300 ~]#
Thanks for the moment
Gianluca
___
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-le...@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/N2J5HCNYLWQAIOIBXO6EKQW57RRTJTDK/
[ovirt-users] Re: OVirt Gluster Fail
During maintenance of a machine the hosted engine crashed. At that point there was no more chance of managing anything. The VMs have paused, and were no longer manageable. I restarted the machine, but one point all the bricks were no longer reachable. Now I am in a situation where the engine support is no longer loaded. The gluster sees the peers connected and the services turned on for the various bricks, but fails to heal the messages that I find for each machine are the following # gluster volume heal engine info Brick 192.170.254.3:/bricks/engine/brick . . . Status: Connected Number of entries: 190 Brick 192.170.254.4:/bricks/engine/brick Status: Il socket di destinazione non è connesso Number of entries: - Brick 192.170.254.6:/bricks/engine/brick Status: Il socket di destinazione non è connesso Number of entries: - this for all the bricks (some have no heal to do because the machines inside were turned off). In practice all the bricks see only localhost as connected. How can I restore the machines? Is there a way to read data from the physical machine and export it so that it can be reused? Unfortunately we need to access that data. Someone can help me. Thanks Andrea ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/EOIY7ZU4GOEMRUNY3CWF6R3JIQNPHLVA/
[ovirt-users] OVirt Gluster Fail
Ho una installazione OVirt 4.1 in gluster. Durante la manutenzione di una macchina abbiamo l'Hosted Engine si è bloccato. A quel punto non c'è stata più possibilità di gestire nulla. Le vm sono andate in pausa, e non sono state più gestibili. Ho atteso il riavvio della macchina, ma a quel punto anche tutti i brick non erano più raggiungibili. Ora mi trovo nella situazione in cui non viene più caricato il mount per l'engine. Il gluster vede i peer connessi e i servizi accesi per i vari brick, ma non riesce a fare l'heal i messaggi che trovo per ogni macchina sono i seguenti: # gluster volume heal engine info Brick 192.170.254.3:/bricks/engine/brick . . . Status: Connected Number of entries: 190 Brick 192.170.254.4:/bricks/engine/brick Status: Il socket di destinazione non è connesso Number of entries: - Brick 192.170.254.6:/bricks/engine/brick Status: Il socket di destinazione non è connesso Number of entries: - questo per tutti i brick (alcuni non hanno alcun heal da fare perchè le macchine all'interno erano state spente). In pratica tutti i brick vedono solo localhost come connesso. Come posso fare per ripristinare le macchine? Esiste un modo per poter leggere i dati dalla macchina fisica ed esportarli in modo da poterli riusare? Purtroppo abbiamo la necessità di accedere a quei dati Qualcuno può aiutarmi. Grazie Andrea ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/M23M4GLKSM3CKWXXSHN3QWF7YHTEEHWW/
[ovirt-users] thanks
You've got knowledge about everything that's why I love this site! Very useful and helpful.. Try browsing my website to gather some information.. Will appreciated it a lot! https://www.nana1004.com / 카지노사이트 ___ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-le...@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BL3FVM3EI62SRAA445GJNEMGOEKQIPMW/
