Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
On Thu, Feb 11, 2016 at 02:50:55PM +0100, Peter Krempa wrote: > > > Whoah. Data corruption accross network? I'm not quite sure > > > whether I'd use this to cover up a problem with the storage > > > technology or network rather than just fix the root cause. If > > > you have 3 copies, and manage to have a sector where all 3 > > > differ then the quorum driver won't help. And it will make it > > > even harder to find any possible problems. > > > > But in that case you detect that it went wrong and you get an I/O > > error. The problem with silent data corruption is that it can be > > hard to detect. > > Yes, and that's why it should be fixed at the network storage > technology layer rather than anywhere else. I've had the chance to discuss this a bit with a cloud provider that is using Quorum. In their experience they have had problems in their tests with Gluster or Ceph, particularly when sharing the same images among several clients. They have experienced major delays and crashes when one of the nodes fail, and in general they don't consider them stable enough for their needs. On the other hand NFS is easy to use and manipulate, robust, and allows the use of hardware appliances. > > If there's a bit-flip across the network Quorum can detect it, > > report it and correct the faulty version without needing to > > rebuild everything. > > I still think that you do wan't to rebulild the whole volume in such > case if you care about your data in the slightest. Otherwise you > don't have to do stuff like this. In general, yes. But that's right, I agree that having API to deal with these scenarios is a good idea and I can work on it. > > > * since we don't use node-names yet, it's not really > > > possible to do block jobs on quorum disks, thus they are > > > forbidden > > > > I'm not sure what's the status of node names in libvirt, I could > > also try to help to make it happen. > > They are basically non-existent. To be honest I think that the node > name support stuff and better approach at constructing block devices > and their backing chains and better handling of block jobs should be > done prior to quorum. > > This series tries to partially do the stuff that is a plan how to > approach some stuff regarding disks. One of them is that the backing > chain of a disk is persisted in the XML and then fully constructed. > > By adding this code the refactor will be even more painful as it > will currently be. > > I'm actually planing to do this in short term future, but > unfortunately this is not a weekend project. > > > > > > * since block jobs are forbidden and rewrite-corrupted can't > > > * be enabled, no way to do the rebuild > > > > 'rewrite-corrupted' can be easily added to the series so I don't > > think that's a problem. The block jobs thing I would need to > > see first. Would you really need to have node names in order to > > rebuild a Quorum? > > Most probably yes. Without them, it will be just an ugly hack. For the common usage I think you can use the device name just fine, but if you have a scenario where a Quorum is part of a backing chain then if wouldn't work without node names. Berto -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
On Wed, Feb 10, 2016 at 17:01:22 +0200, Alberto Garcia wrote: > On Tue, Jan 26, 2016 at 05:36:36PM +0100, Peter Krempa wrote: [...] > > Whoah. Data corruption accross network? I'm not quite sure whether > > I'd use this to cover up a problem with the storage technology or > > network rather than just fix the root cause. If you have 3 copies, > > and manage to have a sector where all 3 differ then the quorum > > driver won't help. And it will make it even harder to find any > > possible problems. > > But in that case you detect that it went wrong and you get an I/O > error. The problem with silent data corruption is that it can be hard > to detect. Yes, and that's why it should be fixed at the network storage technology layer rather than anywhere else. > > If there's a bit-flip across the network Quorum can detect it, > report it and correct the faulty version without needing to rebuild > everything. I still think that you do wan't to rebulild the whole volume in such case if you care about your data in the slightest. Otherwise you don't have to do stuff like this. [...] > > > Quorum is also used for the COLO block replication functionality > > > currently being discussed in QEMU: > > > > > >http://wiki.qemu.org/Features/BlockReplication [...] > Yes, I just wanted to point out one other example of how Quorum is > being used. This current series of Quorum for libvirt is not taking > COLO into account at all, in fact it is still under review in QEMU. Yes and there are apparently some design issues/major problems. If they are going to use this, we should probably wait on the result of that discussion. [...] > > 1) Apart from abusing quorums in fifo mode for COLO I still don't > > think that they are hugely useful. (no, data corruption on NFS > > didn't persuade me) > > It is one of the main reasons why Quorum was written. Here's one more > example of silent data corruption over the network: > > https://cds.cern.ch/record/2026187/files/Adler32_Data_Corruption.pdf That underlines the fact that the network storage protocol does a terrible job in this case. Additionally in the described case the highlighted advantage of adler32 is speed. Patching that with triplicating the data and necessary bandwidth to transfer it does not stack with that really well. Additionally the regular use case remains still broken. > > 2) The implementation in this series as in current state adds a lot > > of code to mintain that wouldn't much used be and is incomplete in > > many aspects: [...] > > * no support for the quorum failure events and reporting > > * no way to control 'rewrite-corrupted' > > I can look into these. > > > * since we don't use node-names yet, it's not really possible to do > > block jobs on quorum disks, thus they are forbidden > > I'm not sure what's the status of node names in libvirt, I could also > try to help to make it happen. They are basically non-existent. To be honest I think that the node name support stuff and better approach at constructing block devices and their backing chains and better handling of block jobs should be done prior to quorum. This series tries to partially do the stuff that is a plan how to approach some stuff regarding disks. One of them is that the backing chain of a disk is persisted in the XML and then fully constructed. By adding this code the refactor will be even more painful as it will currently be. I'm actually planing to do this in short term future, but unfortunately this is not a weekend project. > > > * since block jobs are forbidden and rewrite-corrupted can't be > > * enabled, no way to do the rebuild > > 'rewrite-corrupted' can be easily added to the series so I don't > think that's a problem. The block jobs thing I would need to see > first. Would you really need to have node names in order to rebuild a > Quorum? Most probably yes. Without them, it will be just an ugly hack. Peter signature.asc Description: Digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
On Tue, Jan 26, 2016 at 05:36:36PM +0100, Peter Krempa wrote: > Hi Alberto, Hi and sorry for the late reply, > > I'm the current maintainer of Quorum in QEMU and I'd like to try > > to answer some of your comments. > > > > > So I have a few comments/observations regarding the quorum block > > > driver in qemu and it's usability. > > > > > > At first I'd like to as you to describe your use case a bit > > > more. I'm currently lacking the motivation to do anything about > > > this, as the series is just partial and I don't really see any > > > advantage of using the qorum driver at all and can't come up > > > with any useful use case. > > > > > > Also a good use case is usually a good reason to drive > > > development of a feature and I'm afraid that this could become > > > abandoned without any real use. > > > > The original use case for which Quorum was designed was a data > > center doing redundancy with storage in multiple separate rooms > > shared using NFS. > > There are quite a few existing networked storage cluster solutions, > wouldn't that be a more reasonable option? I don't know all the details of the setup, but as far as I'm aware the goal is to have redundancy and high availability and at the same time keep each one of the servers independent from each other in case the others crash. It's also easier to set up. I can try to get more details if you want. > > One of the issues that the customer was facing was not only > > problems in the file servers themselves but -mainly- data > > corruption accross the network. Quorum can correct this on the fly > > and is able to > > Whoah. Data corruption accross network? I'm not quite sure whether > I'd use this to cover up a problem with the storage technology or > network rather than just fix the root cause. If you have 3 copies, > and manage to have a sector where all 3 differ then the quorum > driver won't help. And it will make it even harder to find any > possible problems. But in that case you detect that it went wrong and you get an I/O error. The problem with silent data corruption is that it can be hard to detect. If there's a bit-flip across the network Quorum can detect it, report it and correct the faulty version without needing to rebuild everything. > > identify which one of the file servers is causing the problem > > without having to rebuild a whole array (like it would be the case > > with RAID). > > Libvirt tries to stay out of doing any usage policy, so this might > be considered a feature. The series needs then polishing to add the > rebuild capability and quorum event handling so that sub-quorate > failed operations are properly reported. > > I think the rebuild is actually a useful in most cases, since it > ensures that all copies are the same. I think the ability to rebuild is in general a good feature, I will look into it and see how to add it to QEMU first. > > Quorum is also used for the COLO block replication functionality > > currently being discussed in QEMU: > > > >http://wiki.qemu.org/Features/BlockReplication > > Oh, so it actually uses the FIFO mode of quorum which I didn't know > about. So basically the quorum driver for COLO serves as a block > duplicator so that one write is sent to the "primary disk" and > second write is sent using nbd to the arbiter rather than using a > blockdev-mirror job. Interresting approach, but COLO stuf was not > really yet considered in libvirt. > > Btw, this series explicitly forbids using less than 2 as vote > threshold. Yes, I just wanted to point out one other example of how Quorum is being used. This current series of Quorum for libvirt is not taking COLO into account at all, in fact it is still under review in QEMU. > > Quorum can fix errors on the fly (there's the 'rewrite-corrupted' > > flag for that), so in those cases no manual intervention is > > required. > > > > If we want a way to auto-resync a complete image that should be > > doable, I believe it's relatively simple to implement in QEMU > > (depending on the semantics). > > > > For the manual resync I also agree that it would be good to > > have a simple API to do that in case the user wants to do it > > manually. That can be done. > > This would be beneficial to have if you don't have > 'rewrite-corrupted' enabled. In that case you want a way to enable > it and then perhaps initiate a full read so that every block gets > checked. Exactly. As I said earlier I think it's a good idea. > > > 4) Necessity for at least 3 copies > > > Since a majority needs to win in a vote, you need at least 3 > > > member disks for this to be fault-tolerant. > > > > > > 5) Lack of speedup > > > Since always all blocks are read from all members and verified > > > the quorum backend doesn't really add any speed to the > > > reads. This can be mostly attributed to the fact that fault > > > tracking is not present. > > > > > > In other cases, due to internal error correcting codes it's very > >
Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
On Wed, Jan 20, 2016 at 16:47:56 +0200, Alberto Garcia wrote: > Hi Peter, Hi Alberto, > > I'm the current maintainer of Quorum in QEMU and I'd like to try to > answer some of your comments. > > On Fri, Jan 08, 2016 at 06:20:04PM +0100, Peter Krempa wrote: > > > So I have a few comments/observations regarding the quorum block > > driver in qemu and it's usability. > > > > At first I'd like to as you to describe your use case a bit > > more. I'm currently lacking the motivation to do anything about > > this, as the series is just partial and I don't really see any > > advantage of using the qorum driver at all and can't come up with > > any useful use case. > > > > Also a good use case is usually a good reason to drive development > > of a feature and I'm afraid that this could become abandoned without > > any real use. > > The original use case for which Quorum was designed was a data center > doing redundancy with storage in multiple separate rooms shared using > NFS. There are quite a few existing networked storage cluster solutions, wouldn't that be a more reasonable option? > One of the issues that the customer was facing was not only problems > in the file servers themselves but -mainly- data corruption accross > the network. Quorum can correct this on the fly and is able to Whoah. Data corruption accross network? I'm not quite sure whether I'd use this to cover up a problem with the storage technology or network rather than just fix the root cause. If you have 3 copies, and manage to have a sector where all 3 differ then the quorum driver won't help. And it will make it even harder to find any possible problems. > identify which one of the file servers is causing the problem without > having to rebuild a whole array (like it would be the case with RAID). Libvirt tries to stay out of doing any usage policy, so this might be considered a feature. The series needs then polishing to add the rebuild capability and quorum event handling so that sub-quorate failed operations are properly reported. I think the rebuild is actually a useful in most cases, since it ensures that all copies are the same. > > Quorum is also used for the COLO block replication functionality > currently being discussed in QEMU: > >http://wiki.qemu.org/Features/BlockReplication Oh, so it actually uses the FIFO mode of quorum which I didn't know about. So basically the quorum driver for COLO serves as a block duplicator so that one write is sent to the "primary disk" and second write is sent using nbd to the arbiter rather than using a blockdev-mirror job. Interresting approach, but COLO stuf was not really yet considered in libvirt. Btw, this series explicitly forbids using less than 2 as vote threshold. > > > 1) No traking of integrity > > As the quorum members don't have headers, failed quorum members > > are not recorded and remembered. The user or management app then > > has to do this externally for given storage devices. > > > > 2) No internal tracking of quorum members > > Members of the quorum don't have any header marking them > > as such and thus any images may be mixed together with > > unforseen/catastrophic results. Higher level management then > > needs to take the role of remembering which images belong > > together. Reimplementing this looks like reimplementing a > > distriuted storage system to me. > > That's right, Quorum does not have its own file format and was > designed to work with any driver or protocol that QEMU supports, so > I'm not sure if there's much that can be done about this. > > > 3) Lack of auto-resync: > > Once the quorum get's few inconsistencies it does not > > automatically resync like the linux MD driver. With the current > > implementation the only way to resync this would be to issue a > > block-mirror (blockCopy) to /dev/null so that all blocks are > > read and rewritten to the identical copy. This also requires a > > user action. > > > > Additionally the member of the quorum is not ignored if it was > > out of sync in any previous time without being resynced allowing > > for split-brain/corruption scenarios. > > Quorum can fix errors on the fly (there's the 'rewrite-corrupted' flag > for that), so in those cases no manual intervention is required. > > If we want a way to auto-resync a complete image that should be > doable, I believe it's relatively simple to implement in QEMU > (depending on the semantics). > > For the manual resync I also agree that it would be good to have a > simple API to do that in case the user wants to do it manually. That > can be done. This would be beneficial to have if you don't have 'rewrite-corrupted' enabled. In that case you want a way to enable it and then perhaps initiate a full read so that every block gets checked. > > > 4) Necessity for at least 3 copies > > Since a majority needs to win in a vote, you need at least 3 > > member disks for this t
Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
Hi Peter, I'm the current maintainer of Quorum in QEMU and I'd like to try to answer some of your comments. On Fri, Jan 08, 2016 at 06:20:04PM +0100, Peter Krempa wrote: > So I have a few comments/observations regarding the quorum block > driver in qemu and it's usability. > > At first I'd like to as you to describe your use case a bit > more. I'm currently lacking the motivation to do anything about > this, as the series is just partial and I don't really see any > advantage of using the qorum driver at all and can't come up with > any useful use case. > > Also a good use case is usually a good reason to drive development > of a feature and I'm afraid that this could become abandoned without > any real use. The original use case for which Quorum was designed was a data center doing redundancy with storage in multiple separate rooms shared using NFS. One of the issues that the customer was facing was not only problems in the file servers themselves but -mainly- data corruption accross the network. Quorum can correct this on the fly and is able to identify which one of the file servers is causing the problem without having to rebuild a whole array (like it would be the case with RAID). Quorum is also used for the COLO block replication functionality currently being discussed in QEMU: http://wiki.qemu.org/Features/BlockReplication > 1) No traking of integrity > As the quorum members don't have headers, failed quorum members > are not recorded and remembered. The user or management app then > has to do this externally for given storage devices. > > 2) No internal tracking of quorum members > Members of the quorum don't have any header marking them > as such and thus any images may be mixed together with > unforseen/catastrophic results. Higher level management then > needs to take the role of remembering which images belong > together. Reimplementing this looks like reimplementing a > distriuted storage system to me. That's right, Quorum does not have its own file format and was designed to work with any driver or protocol that QEMU supports, so I'm not sure if there's much that can be done about this. > 3) Lack of auto-resync: > Once the quorum get's few inconsistencies it does not > automatically resync like the linux MD driver. With the current > implementation the only way to resync this would be to issue a > block-mirror (blockCopy) to /dev/null so that all blocks are > read and rewritten to the identical copy. This also requires a > user action. > > Additionally the member of the quorum is not ignored if it was > out of sync in any previous time without being resynced allowing > for split-brain/corruption scenarios. Quorum can fix errors on the fly (there's the 'rewrite-corrupted' flag for that), so in those cases no manual intervention is required. If we want a way to auto-resync a complete image that should be doable, I believe it's relatively simple to implement in QEMU (depending on the semantics). For the manual resync I also agree that it would be good to have a simple API to do that in case the user wants to do it manually. That can be done. > 4) Necessity for at least 3 copies > Since a majority needs to win in a vote, you need at least 3 > member disks for this to be fault-tolerant. > > 5) Lack of speedup > Since always all blocks are read from all members and verified > the quorum backend doesn't really add any speed to the > reads. This can be mostly attributed to the fact that fault > tracking is not present. > > In other cases, due to internal error correcting codes it's very > unlikely that a storage medium would return a corrupted sector > without producing a error. 4) and 5) are part of the design of Quorum, as I said one the goals is to detect (and correct) silent data corruption on the fly, not to speed up disk access or to be space efficient. > 6) Almost every remote storage technology does quorums internally > Any distributed storage (ceph/rbd, gluster, sheepdog, etc..) > provide the quorum functionality internally with added benefit > that their internal working fixes problems when split of the > network occurs. > > 7) Tools are restricted to qemu and qemu-img > It's a "proprietary" implementation so for a rebuild you have > to use one of the two tools. AFAIK qemu-img is not really > user friendly for the less common disk backends and we don't > really provide any abstraction on top of that. This means > that there really aren't any reasonable tools to do a offline > resync. (Okay, if you know which instance is okay, you can just > copy it ...) Right. If this is important I can propose to write a tool for QEMU to deal with this. It's probably a good idea anyway. > This series also lacks implementation of any user/maganement > warning method that a block operation didn't have 100% votes in the > quorum voting thus
Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
On Thu, Dec 03, 2015 at 15:35:10 +0100, Matthias Gatto wrote: > The purpose of these patches is to introduce quorum for libvirt > I've try to follow this proposal: > http://www.redhat.com/archives/libvir-list/2014-May/msg00533.html > TL;DR: I'm concerned that the quorum implementation is not really useful and will introduce a lot of code with little benefit. --- So I have a few comments/observations regarding the quorum block driver in qemu and it's usability. At first I'd like to as you to describe your use case a bit more. I'm currently lacking the motivation to do anything about this, as the series is just partial and I don't really see any advantage of using the qorum driver at all and can't come up with any useful use case. Also a good use case is usually a good reason to drive development of a feature and I'm afraid that this could become abandoned without any real use. My problems with supporting the quorum backend are: 1) No traking of integrity As the quorum members don't have headers, failed quorum members are not recorded and remembered. The user or management app then has to do this externally for given storage devices. 2) No internal tracking of quorum members Members of the quorum don't have any header marking them as such and thus any images may be mixed together with unforseen/catastrophic results. Higher level management then needs to take the role of remembering which images belong together. Reimplementing this looks like reimplementing a distriuted storage system to me. 3) Lack of auto-resync: Once the quorum get's few inconsistencies it does not automatically resync like the linux MD driver. With the current implementation the only way to resync this would be to issue a block-mirror (blockCopy) to /dev/null so that all blocks are read and rewritten to the identical copy. This also requires a user action. Additionally the member of the quorum is not ignored if it was out of sync in any previous time without being resynced allowing for split-brain/corruption scenarios. 4) Necessity for at least 3 copies Since a majority needs to win in a vote, you need at least 3 member disks for this to be fault-tolerant. 5) Lack of speedup Since always all blocks are read from all members and verified the quorum backend doesn't really add any speed to the reads. This can be mostly attributed to the fact that fault tracking is not present. In other cases, due to internal error correcting codes it's very unlikely that a storage medium would return a corrupted sector without producing a error. 6) Almost every remote storage technology does quorums internally Any distributed storage (ceph/rbd, gluster, sheepdog, etc..) provide the quorum functionality internally with added benefit that their internal working fixes problems when split of the network occurs. 7) Tools are restricted to qemu and qemu-img It's a "proprietary" implementation so for a rebuild you have to use one of the two tools. AFAIK qemu-img is not really user friendly for the less common disk backends and we don't really provide any abstraction on top of that. This means that there really aren't any reasonable tools to do a offline resync. (Okay, if you know which instance is okay, you can just copy it ...) This series also lacks implementation of any user/maganement warning method that a block operation didn't have 100% votes in the quorum voting thus it's not really possible for the users to do a rebuild/diagnostic if something fails. Peter signature.asc Description: Digital signature -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
ping On Thu, Dec 3, 2015 at 3:35 PM, Matthias Gatto wrote: > The purpose of these patches is to introduce quorum for libvirt > I've try to follow this proposal: > http://www.redhat.com/archives/libvir-list/2014-May/msg00533.html > > This feature ask for 5 task: > > 1) Allow a _virStorageSource to contain more than one backing store. > > Because all the actual libvirt code use the backingStore field > as a pointer and we needs want to change that, I've decide to encapsulate > the backingStore field to simplifie the array manipulation. > > 2) Add the missing field a quorum need in _virStorageSource and > the VIR_STORAGE_TYPE_QUORUM and VIR_STORAGE_FILE_QUORUM in > their respectives enums. > > 3) Parse and format the xml > Because a quorum allows to have more than one backing store at the same level > we need to change virDomainDiskDefFormat and virDomainDiskDefParseXML > to call virDomainDiskBackingStoreFormat and virDomainDiskBackingStoreParse > in a loop. > virDomainDiskBackingStoreFormat and virDomainDiskBackingStoreParse can > call themself recursively in a loop because a quorum can contain another > quorum > > 4) Build qemu string > As for the xml, we have to call the function which create quorum recursively. > But this task have the problem explained here: > http://www.redhat.com/archives/libvir-list/2014-October/msg00529.html > The _virStorageSource missing some informations that can be passed to > a child, and therefore this version of quorum is incomplet. > > 5) Allow to hotplug/change a disk in a quorum > This part is not present in these patches because for this task > we have to use blockdev-add, and currently libvirt use > device_add for hotpluging that doesn't allow to hotplug quorum childs. > > > V2: > -Rebase on master > -Add Documentation > > V3: > -Transforme the backingStore field in virStorageSource into > an array of pointer instead of a pointer > -Modify virStorageSourceSetBackingStore to allow it to expand > the backingStore size. > > V4: > -Rebase on master > > V5: > -Rebase on master > -patch 1-4/9: use patchs from John Ferlan > -patch 4/9: check return of virStorageSourceSetBackingStore > -patch 5/9: report type of error on virStorageSourceSetBackingStore > > v6: > -Rebase on master > -Remove node-name patch > -patch 06/13: Add virStorageSourceIsRAID > -patch 10/13: Add virDomainDefHasRAID > -patch 11-13/13: Block all unsupported operations > > v7: > -Rebase on master. > -Parse unconditionally backing store. > -fold qemuBuildRAIDFileSourceStr into qemuBuildRAIDStr. > -use 0/-1 return values when failing instead of bool. > -virStorageSourceSetBackingStore now free backing store when they are > already set. > > Matthias Gatto (13): > virstoragefile: Add virStorageSourceGetBackingStore > virstoragefile: Always use virStorageSourceGetBackingStore to get > backing store > virstoragefile: Add virStorageSourceSetBackingStore > virstoragefile: Always use virStorageSourceSetBackingStore to set > backing store > virstoragefile: change backingStore to backingStores. > virstoragefile: Add virStorageSourceIsRAID > virstoragefile: Add quorum in virstoragefile > domain_conf: Read and Write quorum config > qemu: Add quorum support in qemuBuildDriveDevStr > domain: Add virDomainDefHasRAID > qemu: Forbid blocks operations with quorum > qemu: qemuDomainGetBlockInfo quorum support > qemu: qemuDiskPathToAlias quorum support > > docs/formatdomain.html.in | 23 - > docs/schemas/domaincommon.rng | 21 +++- > docs/schemas/storagecommon.rng| 1 + > docs/schemas/storagevol.rng | 1 + > src/conf/domain_conf.c| 174 > -- > src/conf/domain_conf.h| 1 + > src/conf/storage_conf.c | 23 +++-- > src/libvirt_private.syms | 4 + > src/qemu/qemu_cgroup.c| 4 +- > src/qemu/qemu_command.c | 78 +++ > src/qemu/qemu_domain.c| 2 +- > src/qemu/qemu_driver.c| 72 ++ > src/qemu/qemu_migration.c | 1 + > src/qemu/qemu_monitor_json.c | 4 +- > src/security/security_dac.c | 2 +- > src/security/security_selinux.c | 4 +- > src/security/virt-aa-helper.c | 2 +- > src/storage/storage_backend.c | 24 +++-- > src/storage/storage_backend_fs.c | 45 + > src/storage/storage_backend_gluster.c | 11 ++- > src/storage/storage_backend_logical.c | 15 ++- > src/storage/storage_driver.c | 3 +- > src/util/virstoragefile.c | 121 --- > src/util/virstoragefile.h | 15 ++- > tests/virstoragetest.c| 18 ++-- > 25 files changed, 518 insertions(+), 151 d
[libvirt] [PATCH v7 00/13] qemu: Add quorum support to libvirt
The purpose of these patches is to introduce quorum for libvirt I've try to follow this proposal: http://www.redhat.com/archives/libvir-list/2014-May/msg00533.html This feature ask for 5 task: 1) Allow a _virStorageSource to contain more than one backing store. Because all the actual libvirt code use the backingStore field as a pointer and we needs want to change that, I've decide to encapsulate the backingStore field to simplifie the array manipulation. 2) Add the missing field a quorum need in _virStorageSource and the VIR_STORAGE_TYPE_QUORUM and VIR_STORAGE_FILE_QUORUM in their respectives enums. 3) Parse and format the xml Because a quorum allows to have more than one backing store at the same level we need to change virDomainDiskDefFormat and virDomainDiskDefParseXML to call virDomainDiskBackingStoreFormat and virDomainDiskBackingStoreParse in a loop. virDomainDiskBackingStoreFormat and virDomainDiskBackingStoreParse can call themself recursively in a loop because a quorum can contain another quorum 4) Build qemu string As for the xml, we have to call the function which create quorum recursively. But this task have the problem explained here: http://www.redhat.com/archives/libvir-list/2014-October/msg00529.html The _virStorageSource missing some informations that can be passed to a child, and therefore this version of quorum is incomplet. 5) Allow to hotplug/change a disk in a quorum This part is not present in these patches because for this task we have to use blockdev-add, and currently libvirt use device_add for hotpluging that doesn't allow to hotplug quorum childs. V2: -Rebase on master -Add Documentation V3: -Transforme the backingStore field in virStorageSource into an array of pointer instead of a pointer -Modify virStorageSourceSetBackingStore to allow it to expand the backingStore size. V4: -Rebase on master V5: -Rebase on master -patch 1-4/9: use patchs from John Ferlan -patch 4/9: check return of virStorageSourceSetBackingStore -patch 5/9: report type of error on virStorageSourceSetBackingStore v6: -Rebase on master -Remove node-name patch -patch 06/13: Add virStorageSourceIsRAID -patch 10/13: Add virDomainDefHasRAID -patch 11-13/13: Block all unsupported operations v7: -Rebase on master. -Parse unconditionally backing store. -fold qemuBuildRAIDFileSourceStr into qemuBuildRAIDStr. -use 0/-1 return values when failing instead of bool. -virStorageSourceSetBackingStore now free backing store when they are already set. Matthias Gatto (13): virstoragefile: Add virStorageSourceGetBackingStore virstoragefile: Always use virStorageSourceGetBackingStore to get backing store virstoragefile: Add virStorageSourceSetBackingStore virstoragefile: Always use virStorageSourceSetBackingStore to set backing store virstoragefile: change backingStore to backingStores. virstoragefile: Add virStorageSourceIsRAID virstoragefile: Add quorum in virstoragefile domain_conf: Read and Write quorum config qemu: Add quorum support in qemuBuildDriveDevStr domain: Add virDomainDefHasRAID qemu: Forbid blocks operations with quorum qemu: qemuDomainGetBlockInfo quorum support qemu: qemuDiskPathToAlias quorum support docs/formatdomain.html.in | 23 - docs/schemas/domaincommon.rng | 21 +++- docs/schemas/storagecommon.rng| 1 + docs/schemas/storagevol.rng | 1 + src/conf/domain_conf.c| 174 -- src/conf/domain_conf.h| 1 + src/conf/storage_conf.c | 23 +++-- src/libvirt_private.syms | 4 + src/qemu/qemu_cgroup.c| 4 +- src/qemu/qemu_command.c | 78 +++ src/qemu/qemu_domain.c| 2 +- src/qemu/qemu_driver.c| 72 ++ src/qemu/qemu_migration.c | 1 + src/qemu/qemu_monitor_json.c | 4 +- src/security/security_dac.c | 2 +- src/security/security_selinux.c | 4 +- src/security/virt-aa-helper.c | 2 +- src/storage/storage_backend.c | 24 +++-- src/storage/storage_backend_fs.c | 45 + src/storage/storage_backend_gluster.c | 11 ++- src/storage/storage_backend_logical.c | 15 ++- src/storage/storage_driver.c | 3 +- src/util/virstoragefile.c | 121 --- src/util/virstoragefile.h | 15 ++- tests/virstoragetest.c| 18 ++-- 25 files changed, 518 insertions(+), 151 deletions(-) -- 2.6.2 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list