----- Original Message ----- > From: "Jiri Moskovcak" <jmosk...@redhat.com> > To: "Nir Soffer" <nsof...@redhat.com> > Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon > Mureinik" <amure...@redhat.com>, "Greg > Padgett" <gpadg...@redhat.com>, "Doron Fediuck" <dfedi...@redhat.com> > Sent: Wednesday, May 7, 2014 10:21:28 AM > Subject: Re: [ovirt-devel] vdsm disabling logical volumes > > On 05/05/2014 03:19 PM, Nir Soffer wrote: > > ----- Original Message ----- > >> From: "Jiri Moskovcak" <jmosk...@redhat.com> > >> To: "Nir Soffer" <nsof...@redhat.com> > >> Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon > >> Mureinik" <amure...@redhat.com>, "Greg > >> Padgett" <gpadg...@redhat.com> > >> Sent: Monday, May 5, 2014 3:44:21 PM > >> Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >> > >> On 05/05/2014 02:37 PM, Nir Soffer wrote: > >>> ----- Original Message ----- > >>>> From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>> To: "Nir Soffer" <nsof...@redhat.com> > >>>> Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon > >>>> Mureinik" <amure...@redhat.com>, "Greg > >>>> Padgett" <gpadg...@redhat.com> > >>>> Sent: Monday, May 5, 2014 3:16:37 PM > >>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >>>> > >>>> On 05/05/2014 12:01 AM, Nir Soffer wrote: > >>>>> ----- Original Message ----- > >>>>>> From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>>>> To: "Nir Soffer" <nsof...@redhat.com> > >>>>>> Cc: devel@ovirt.org > >>>>>> Sent: Sunday, May 4, 2014 9:23:49 PM > >>>>>> Subject: Re: [ovirt-devel] vdsm disabling logical volumes > >>>>>> > >>>>>> On 05/04/2014 07:57 PM, Nir Soffer wrote: > >>>>>>> ----- Original Message ----- > >>>>>>>> From: "Jiri Moskovcak" <jmosk...@redhat.com> > >>>>>>>> To: devel@ovirt.org > >>>>>>>> Sent: Sunday, May 4, 2014 8:08:33 PM > >>>>>>>> Subject: [ovirt-devel] vdsm disabling logical volumes > >>>>>>>> > >>>>>>>> Greetings vdsm developers! > >>>>>>>> > >>>>>>>> While working on adding ISCSI support to the hosted engine tools, I > >>>>>>>> ran > >>>>>>>> into a problem with vdms. It seems that when stopped vdsm > >>>>>>>> deactivates > >>>>>>>> ALL logical volumes in it's volume group and when it starts it > >>>>>>>> reactivates only specific logical volumes. This is a problem for > >>>>>>>> hosted > >>>>>>>> engine tools as they create logical volumes in the same volume group > >>>>>>>> and > >>>>>>>> when vdsm deactivates the LVs the hosted engine tools don't have a > >>>>>>>> way > >>>>>>>> to reactivate it, because the services drop the root permissions and > >>>>>>>> are > >>>>>>>> running as vdsm and apparently only root can activate LVs. > >>>>>>> > >>>>>>> Can you describe what volumes are you creating, and why? > >>>>>> > >>>>>> We create hosted-engine.lockspace (for sanlock) and > >>>>>> hosted-engine.metadata (keeps data about hosted engine hosts) > >>>>> > >>>>> Do you create these lvs in every vdsm vg? > >>>> > >>>> - only in the first vg created by vdsm while deploying hosted-engine > > > > It seems that the hosted engine has single point of failure - the random > > vg that contains hosted engine data. > > > >>>> > >>>>> Is this part of the domain structure > >>>>> used by hosted engine, or it has nothing to do with the storage domain? > >>>> > >>>> - sorry, I don't understand this question. How can I tell if it has > >>>> something to do with the storage domain? It's for storing data about > >>>> hosts set up to run the hosted-engine and data about state of engine and > >>>> the state of VM running the engine. > >>> > >>> Can you tell us exactly what lvs you are creating, and on which vg? > >>> > >>> And how are you creating those lvs - I guess through vdsm? > >>> > >> > >> - no hosted-engine tools do that by calling: > >> > >> lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE, > >> stderr=subprocess.PIPE, > >> args=["lvm", "lvcreate", "-L", str(size_bytes)+"B", > >> "-n", lv_name, vg_uuid]) > >> .. > > > > How do you ensure that another host is not modifying the same vg in the > > same time? > > > > If you are not ensuring this, you will corrupt this vg sooner or later. > > > > When a storage domain is detached from a host, for example when the host > > is in maintenance mode, lvs on the shared storage may be deleted, > > invalidating > > the devices mapper maps for these devices. If you write to an lv with wrong > > maps, you may be writing to an extent belonging to another lv, corrupting > > that > > lv data, or even worse corrupting the engine vg data. > > > > How do you ensure that the lvs are not deleted while you are using them? > > > >> > >>> The output of lvs command on a host with hosted engine installed will > >>> help us to understand what you are doing, and then we can think more > >>> clearly > >>> what would be the best way to support this in vdsm. > >> > >> The output of lvs: http://fpaste.org/99196/93619139/ > >> > >> HE created these two LVs: > >> ha_agent-hosted-engine.lockspace > >> ha_agent-hosted-engine.metadata > > > > Why do you create these lvs on a vg owned by vdsm? > > > > If you want total control of these lvs, I suggest that you create your own > > vg and put what ever lvs you like there. > > > > I would rather not go this way (at least not for 3.5) as it's too much > code changes in hosted-engine. On the other hand the logic in vdsm seems > wrong because it's not complementary (disabling all LVs and then > enabling just some of them) and should be fixed anyway. This problem is > blocking one of our 3.5 features so I've created rhbz#1094657 to track it.
Can you elaborate on this? How should vdsm behave better, and why? > > --Jirka > > >> > >> --Jirka > >>> > >>>> > >>>>> > >>>>>> > >>>>>>> > >>>>>>>> So far the > >>>>>>>> only suitable solution seems to be to change vdsm to only > >>>>>>>> deactivate/activate it's own LVs. > >>>>>>> > >>>>>>> This sounds reasonable. You can add a list of hosted engine lv names > >>>>>>> and skip these volumes when deactivating vdsm volumes. > >>>>>> > >>>>>> - this sounds a bit suboptimal, vdsm already has list of it's LVs, so > >>>>>> it > >>>>>> can just disable only LVs known to it, otherwise we would have to > >>>>>> change > >>>>>> the list everytime we add some LV to the group > >>>>> > >>>>> vdsm has a list of special lvs, that needs special treatment. > >>>>> Otherwise, > >>>>> it > >>>>> consider any other lv as owned by vdsm, and will deactivate them when > >>>>> they > >>>>> are > >>>>> not used. > >>>>> > >>>>> I agree that this will create a dependency, but this can also be > >>>>> solved. > >>>>> For example, vdsm can load the list from a file installed by hosted > >>>>> engine, > >>>>> like the typical conf.d directories. > >>>>> > >>>> > >>>> - ok, this is something I actually don't have strong opinion about, for > >>>> me adding a file with a list of LVs or tagging the logical volumes is > >>>> almost the same, I just need a way to tell vdsm which LVs to ignore.. > >>>> > >>>>>> > >>>>>>> > >>>>>>> Another solution is to tag hosted engine lvs, and have vdsm ignore > >>>>>>> lvs that contains this tag. > >>>>>> > >>>>>> - this sounds good, because if we teach vdsm to ignore LVs with some > >>>>>> tag > >>>>>> we can add new LVs in future without changing vdsm. This however > >>>>>> applies > >>>>>> also to the solution where vdsm only disables it's own LVs, > >>>>> > >>>>> vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like > >>>>> this > >>>>> using some historic tags we keep (e.g. RHAT_*), but I'd rather add new > >>>>> tag > >>>>> with > >>>>> clear semantic than use some random historic value we have. > >>>>> > >>>>>> so it > >>>>>> depends on vdsm devels which solution they find better. I think the > >>>>>> solution without tags is better, because is simpler and others (like > >>>>>> hosted-engine) can just createlv and don't bother with tags.. > >>>>> > >>>>> I think that a generic tag like OVIRT_IGNORE is an easy solution for > >>>>> everyone. > >>>>> > >>>>> Federico, what do you think? > >>>>> > >>>>> Nir > >>>>> > >>>> > >>>> > >> > >> > > _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel