+list, sorry for dropping it ---------- Forwarded message ---------- From: "Guido Trotter" <[email protected]> Date: 25 Jan 2013 14:40 Subject: Re: RFC: ganeti improvements To: "Constantinos Venetsanopoulos" <[email protected]> Cc:
On Fri, Jan 25, 2013 at 12:52 PM, Constantinos Venetsanopoulos <[email protected]> wrote: > Hello all, > > > On 01/25/2013 01:42 PM, Guido Trotter wrote: >> >> On Fri, Jan 25, 2013 at 11:01 AM, Nicolas Sebrecht <[email protected]> >> wrote: >>> >>> [ X-post to ganeti and ganeti-devel mailing lists. ] >>> >>> >>> Hi all, >>> >>> Ganeti is now used in our compagny, internally. We'd like to make it >>> full solution ready for our customers but we are missing some critical >>> features. >>> >>> Before going further and starting to play with the code I'd like to >>> share my thoughts for comments. The idea is to submit patches in the >>> correct direction as early as possible in the development process. >>> >>> I didn't looked much at the code, yet. >>> >>> >>> 1. Improving the KVM integration. >>> >>> 1.1. Hibernation. >>> >>> AFAICT, we can't hibernate a KVM instance from Ganeti. I guess it is >>> possible to do with QMP and store the RAM in a dedicated path or LVM >>> volume. >>> >> I believe adding a gnt-instance hibernate (or shutdown --hibernate) >> would work well from our side. > > > I agree on that. > > >>> 1.2. CPU Tunning. >>> >>> - CPU pinning (mask) is supported. >>> - CPU SMP is under work. >>> >> Yes, this is all part of 2.7. Feel free to test it further and let us >> know if it works. >> >>> 2. Snapshots. >>> >>> The current gnt-backup tool does not fit our needs. Here are some >>> limitations I've found (correct me if I'm wrong): >>> >>> - It relies on os-type (and variants?) which is fine if you have >>> homogeneous instances but does not help for a lot of per-instance >>> customing. >>> BTW, os-interface mix various kind of requirements (creation, >>> export/import, rename, verify). It is hard for us to get relevant >>> families of instances matching all these kind of requirements. >>> >> We do want to improve the os area, and are willing of course to >> discuss a design and accept patches. >> While allowing an OS to do something smarter is good for disk space >> and efficiency, we can definitely add a fallback option. > > > Comments below. > > >>> - gnt-backup command does not support more than one export strategy. >>> >> What export strategies would you like? >> >>> - The export scripts are too much hard do understand and modify for our >>> system administrators. But full instances management must be >>> accessible for sysadmins (creation, tunning, backups, etc). >>> >> They don't seem "very" hard, but I definitely agree to any >> simplification, and to making sure these features can run outside the >> node os, if possible. (a-la-snf-image, but ganeti driven rather than >> os driven, perhaps?) > > > Comments below. > > >>> - Perhaps other limitations I'm not aware? >>> >> The oses running as root on the node are a limitation if you want >> arbitrary users to upload their own. >> snf-image solves it for kvm, but it would be good to have ganeti >> understand the concept of "boot auxiliary image to drive os >> installation and other events". >> >>> I believe it might worth starting another tool from scratch with a new >>> design. I'm not saying it would have to replace the current >>> gnt-backup. I'm more suggesting something like 'gnt-snapshot' or >>> whatever at the side. Any thoughts on this? >>> >> I believe having gnt-backup snapshot would be as good. I don't see a >> particular reason for another extra command (sometimes I wonder why it >> should be gnt-backup at all, rather than subcommands of gnt-instance). > > > Comments below. > > >>> 3. Clone. >>> >>> Clonning an instance looks not supported. >>> >> Marco's idea sounds good, on this. Why not gnt-backup clone that does >> this via export+import? (if not somehow in a smarter way?) > > > The ability for clones and snapshots is a very hot issue, but from our > experience, > it's not that easy to tackle, if you want to do it correctly. I believe that > it > should not be confused with the OS definitions level and the gnt-backup > functionality. Some food for thought: > > Currently, we are able to do full clones and snapshots using a custom > ExtStorage > provider. This provider speaks with our custom distributed storage layer > which > undertakes the actual operation. So, to create an instance by cloning an > existing > image disk known by the external system, we run: > > gnt-instance add -t ext -o snf-image+default ... > --disk > 0:size=10G,origin="disk-UUID-known-to-the-external-system" > > This way "origin" is passed to the ExtStorage provider as a parameter. The > provider knows from where and how to clone the disk transparently to > Ganeti. After that, the OS definition (snf-image) operates on a top of the > disk > and does all the customization (sets passwords, resizes the filesystem, > etc). > > However, we believe (and after a live talk with Iustin) that it would be > nice > to have Ganeti know about the concept of cloning and snapshotting. But to > do so, I believe we would need the following: > > * Disk objects should become logically detached from instances, obtain their > own UUIDs (as has happened with networks and nodegroups) and exist > independently of the instances. One should be able to have disks that are > not attached to any instance. > > * The cloning and snapshotting functionality should be implemented at the > bdev.py level as separate functions for every template and not at the > OS level. > Note that these 2 are not necessarily interdependent, and you can have 2 without 1. It would be possible to start by implementing cloning and unmanaged snapshotting at the bdev level, and use still an instance as the target of cloning, and then later on do the actual separation of storage entities management and better management of the snapshots. > The above would allow for very neat things apart from cloning/snapshotting, > such as volume attachment, detachment etc. This functionality, combined > with Storage Pools (comments below :) ) would open up a whole new > perspective to what Ganeti can do. > ACK, what do you think of the plan of doing it in two stages, as outlined above? (to have the functionality earlier and also not to make it depend on something quite big, but actually vice versa to build the detachment on the grounds of it) > > >>> 4. Different disk types for an instance. >>> >>> I've heard some discussions about that but can't remember what's the >>> current status. Not urgent at all but I guess this is something we will >>> need some day. >>> >> There is a design for storage pools, but it's not being implemented >> yet. We definitely would like to have it in. > > > Storage Pools are coming :) After merging the ExtStorage patch series, > this is the next to come. During the ExtStorage merge we had some > interesting discussions with Iustin that have to do with Storage Pools, > so I'll try to wrap everything up and revive the discussion on their > design as soon as possible. > ACK, thanks a lot for your help on this. Guido --
