On 05/05/2014 03:19 PM, Nir Soffer wrote:
----- Original Message -----
From: "Jiri Moskovcak" <jmosk...@redhat.com>
To: "Nir Soffer" <nsof...@redhat.com>
Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon Mureinik"
<amure...@redhat.com>, "Greg
Padgett" <gpadg...@redhat.com>
Sent: Monday, May 5, 2014 3:44:21 PM
Subject: Re: [ovirt-devel] vdsm disabling logical volumes
On 05/05/2014 02:37 PM, Nir Soffer wrote:
----- Original Message -----
From: "Jiri Moskovcak" <jmosk...@redhat.com>
To: "Nir Soffer" <nsof...@redhat.com>
Cc: devel@ovirt.org, "Federico Simoncelli" <fsimo...@redhat.com>, "Allon
Mureinik" <amure...@redhat.com>, "Greg
Padgett" <gpadg...@redhat.com>
Sent: Monday, May 5, 2014 3:16:37 PM
Subject: Re: [ovirt-devel] vdsm disabling logical volumes
On 05/05/2014 12:01 AM, Nir Soffer wrote:
----- Original Message -----
From: "Jiri Moskovcak" <jmosk...@redhat.com>
To: "Nir Soffer" <nsof...@redhat.com>
Cc: devel@ovirt.org
Sent: Sunday, May 4, 2014 9:23:49 PM
Subject: Re: [ovirt-devel] vdsm disabling logical volumes
On 05/04/2014 07:57 PM, Nir Soffer wrote:
----- Original Message -----
From: "Jiri Moskovcak" <jmosk...@redhat.com>
To: devel@ovirt.org
Sent: Sunday, May 4, 2014 8:08:33 PM
Subject: [ovirt-devel] vdsm disabling logical volumes
Greetings vdsm developers!
While working on adding ISCSI support to the hosted engine tools, I
ran
into a problem with vdms. It seems that when stopped vdsm deactivates
ALL logical volumes in it's volume group and when it starts it
reactivates only specific logical volumes. This is a problem for
hosted
engine tools as they create logical volumes in the same volume group
and
when vdsm deactivates the LVs the hosted engine tools don't have a way
to reactivate it, because the services drop the root permissions and
are
running as vdsm and apparently only root can activate LVs.
Can you describe what volumes are you creating, and why?
We create hosted-engine.lockspace (for sanlock) and
hosted-engine.metadata (keeps data about hosted engine hosts)
Do you create these lvs in every vdsm vg?
- only in the first vg created by vdsm while deploying hosted-engine
It seems that the hosted engine has single point of failure - the random
vg that contains hosted engine data.
Is this part of the domain structure
used by hosted engine, or it has nothing to do with the storage domain?
- sorry, I don't understand this question. How can I tell if it has
something to do with the storage domain? It's for storing data about
hosts set up to run the hosted-engine and data about state of engine and
the state of VM running the engine.
Can you tell us exactly what lvs you are creating, and on which vg?
And how are you creating those lvs - I guess through vdsm?
- no hosted-engine tools do that by calling:
lvc = popen(stdin=subprocess.PIPE, stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
args=["lvm", "lvcreate", "-L", str(size_bytes)+"B",
"-n", lv_name, vg_uuid])
..
How do you ensure that another host is not modifying the same vg in the
same time?
If you are not ensuring this, you will corrupt this vg sooner or later.
When a storage domain is detached from a host, for example when the host
is in maintenance mode, lvs on the shared storage may be deleted, invalidating
the devices mapper maps for these devices. If you write to an lv with wrong
maps, you may be writing to an extent belonging to another lv, corrupting that
lv data, or even worse corrupting the engine vg data.
How do you ensure that the lvs are not deleted while you are using them?
The output of lvs command on a host with hosted engine installed will
help us to understand what you are doing, and then we can think more
clearly
what would be the best way to support this in vdsm.
The output of lvs: http://fpaste.org/99196/93619139/
HE created these two LVs:
ha_agent-hosted-engine.lockspace
ha_agent-hosted-engine.metadata
Why do you create these lvs on a vg owned by vdsm?
If you want total control of these lvs, I suggest that you create your own
vg and put what ever lvs you like there.
I would rather not go this way (at least not for 3.5) as it's too much
code changes in hosted-engine. On the other hand the logic in vdsm seems
wrong because it's not complementary (disabling all LVs and then
enabling just some of them) and should be fixed anyway. This problem is
blocking one of our 3.5 features so I've created rhbz#1094657 to track it.
--Jirka
--Jirka
So far the
only suitable solution seems to be to change vdsm to only
deactivate/activate it's own LVs.
This sounds reasonable. You can add a list of hosted engine lv names
and skip these volumes when deactivating vdsm volumes.
- this sounds a bit suboptimal, vdsm already has list of it's LVs, so it
can just disable only LVs known to it, otherwise we would have to change
the list everytime we add some LV to the group
vdsm has a list of special lvs, that needs special treatment. Otherwise,
it
consider any other lv as owned by vdsm, and will deactivate them when
they
are
not used.
I agree that this will create a dependency, but this can also be solved.
For example, vdsm can load the list from a file installed by hosted
engine,
like the typical conf.d directories.
- ok, this is something I actually don't have strong opinion about, for
me adding a file with a list of LVs or tagging the logical volumes is
almost the same, I just need a way to tell vdsm which LVs to ignore..
Another solution is to tag hosted engine lvs, and have vdsm ignore
lvs that contains this tag.
- this sounds good, because if we teach vdsm to ignore LVs with some tag
we can add new LVs in future without changing vdsm. This however applies
also to the solution where vdsm only disables it's own LVs,
vdsm own lvs are *all* lvs in vdsm vgs. We can implement something like
this
using some historic tags we keep (e.g. RHAT_*), but I'd rather add new
tag
with
clear semantic than use some random historic value we have.
so it
depends on vdsm devels which solution they find better. I think the
solution without tags is better, because is simpler and others (like
hosted-engine) can just createlv and don't bother with tags..
I think that a generic tag like OVIRT_IGNORE is an easy solution for
everyone.
Federico, what do you think?
Nir
_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
