On Wed, May 07, 2014 at 09:56:03AM +0200, Jiri Moskovcak wrote: > On 05/07/2014 09:28 AM, Nir Soffer wrote: > >----- Original Message ----- > >>From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>To: "Nir Soffer" <nsof...@redhat.com> > >>Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon > >>Mureinik" <amure...@redhat.com>, "Greg > >>Padgett" <gpadg...@redhat.com>, "Doron Fediuck" <dfedi...@redhat.com> > >>Sent: Wednesday, May 7, 2014 10:21:28 AM > >>Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >> > >>On 05/05/2014 03:19 PM, Nir Soffer wrote: > >>>----- Original Message ----- > >>>>From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>>To: "Nir Soffer" <nsof...@redhat.com> > >>>>Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon > >>>>Mureinik" <amure...@redhat.com>, "Greg > >>>>Padgett" <gpadg...@redhat.com> > >>>>Sent: Monday, May 5, 2014 3:44:21 PM > >>>>Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >>>> > >>>>On 05/05/2014 02:37 PM, Nir Soffer wrote: > >>>>>----- Original Message ----- > >>>>>>From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>>>>To: "Nir Soffer" <nsof...@redhat.com> > >>>>>>Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon > >>>>>>Mureinik" <amure...@redhat.com>, "Greg > >>>>>>Padgett" <gpadg...@redhat.com> > >>>>>>Sent: Monday, May 5, 2014 3:16:37 PM > >>>>>>Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >>>>>> > >>>>>>On 05/05/2014 12:01 AM, Nir Soffer wrote: > >>>>>>>----- Original Message ----- > >>>>>>>>From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>>>>>>To: "Nir Soffer" <nsof...@redhat.com> > >>>>>>>>Cc: devel@ovirt.org > >>>>>>>>Sent: Sunday, May 4, 2014 9:23:49 PM > >>>>>>>>Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >>>>>>>> > >>>>>>>>On 05/04/2014 07:57 PM, Nir Soffer wrote: > >>>>>>>>>----- Original Message ----- > >>>>>>>>>>From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>>>>>>>>To: devel@ovirt.org > >>>>>>>>>>Sent: Sunday, May 4, 2014 8:08:33 PM > >>>>>>>>>>Subject: [ovirt-devel] vdsm disabling logical volumes > >>>>>>>>>> > >>>>>>>>>>Greetings vdsm developers! > >>>>>>>>>> > >>>>>>>>>>While working on adding ISCSI support to the hosted engine tools, I > >>>>>>>>>>ran > >>>>>>>>>>into a problem with vdms. It seems that when stopped vdsm > >>>>>>>>>>deactivates > >>>>>>>>>>ALL logical volumes in it's volume group and when it starts it > >>>>>>>>>>reactivates only specific logical volumes. This is a problem for > >>>>>>>>>>hosted > >>>>>>>>>>engine tools as they create logical volumes in the same volume group > >>>>>>>>>>and > >>>>>>>>>>when vdsm deactivates the LVs the hosted engine tools don't have a > >>>>>>>>>>way > >>>>>>>>>>to reactivate it, because the services drop the root permissions and > >>>>>>>>>>are > >>>>>>>>>>running as vdsm and apparently only root can activate LVs. > >>>>>>>>> > >>>>>>>>>Can you describe what volumes are you creating, and why? > >>>>>>>> > >>>>>>>>We create hosted-engine.lockspace (for sanlock) and > >>>>>>>>hosted-engine.metadata (keeps data about hosted engine hosts) > >>>>>>> > >>>>>>>Do you create these lvs in every vdsm vg? > >>>>>> > >>>>>>- only in the first vg created by vdsm while deploying hosted-engine > >>> > >>>It seems that the hosted engine has single point of failure - the random > >>>vg that contains hosted engine data. > >>> > >>>>>> > >>>>>>>Is this part of the domain structure > >>>>>>>used by hosted engine, or it has nothing to do with the storage domain? > >>>>>> > >>>>>>- sorry, I don't understand this question. How can I tell if it has > >>>>>>something to do with the storage domain? It's for storing data about > >>>>>>hosts set up to run the hosted-engine and data about state of engine and > >>>>>>the state of VM running the engine. > >>>>> > >>>>>Can you tell us exactly what lvs you are creating, and on which vg? > >>>>> > >>>>>And how are you creating those lvs - I guess through vdsm? > >>>>> > >>>> > >>>>- no hosted-engine tools do that by calling: > >>>> > >>>>lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, > >>>> stderr=subprocess.PIPE, > >>>> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", > >>>> "-n", lv_name, vg_uuid]) > >>>>.. > >>> > >>>How do you ensure that another host is not modifying the same vg in the > >>>same time? > >>> > >>>If you are not ensuring this, you will corrupt this vg sooner or later. > >>> > >>>When a storage domain is detached from a host, for example when the host > >>>is in maintenance mode, lvs on the shared storage may be deleted, > >>>invalidating > >>>the devices mapper maps for these devices. If you write to an lv with wrong > >>>maps, you may be writing to an extent belonging to another lv, corrupting > >>>that > >>>lv data, or even worse corrupting the engine vg data. > >>> > >>>How do you ensure that the lvs are not deleted while you are using them? > >>> > >>>> > >>>>>The output of lvs command on a host with hosted engine installed will > >>>>>help us to understand what you are doing, and then we can think more > >>>>>clearly > >>>>>what would be the best way to support this in vdsm. > >>>> > >>>>The output of lvs: http://fpaste.org/99196/93619139/ > >>>> > >>>>HE created these two LVs: > >>>>ha_agent-hosted-engine.lockspace > >>>>ha_agent-hosted-engine.metadata > >>> > >>>Why do you create these lvs on a vg owned by vdsm? > >>> > >>>If you want total control of these lvs, I suggest that you create your own > >>>vg and put what ever lvs you like there. > >>> > >> > >>I would rather not go this way (at least not for 3.5) as it's too much > >>code changes in hosted-engine. On the other hand the logic in vdsm seems > >>wrong because it's not complementary (disabling all LVs and then > >>enabling just some of them) and should be fixed anyway. This problem is > >>blocking one of our 3.5 features so I've created rhbz#1094657 to track it. > > > >Can you elaborate on this? How should vdsm behave better, and why? > > Sure. So far I didn't hear any reason why it behaves like this and > it seems not logical to disable *all* and then enable just *some*. > > How: Disabling and enabling operations should be complementary. > Why: To be less surprising.
There is an asymmetry between activation and deactivation of an LV. A mistakenly-active LV can cause data corruption. Making sure that this does not happen is more important than a new feature. We do not want to deactivate and then re-activating the same set of LVs. That would be illogical. We intentionally deactivate LVs that are no longer used on the specific host - that's important if a qemu died while Vdsm was down, leaving a stale LV behind. Design-wise, Vdsm would very much like to keep its ownership of Vdsm-created storage domain. Let us discuss how your feature can be implemented without this breach of ownership. _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
