On Fri, Jan 25, 2013 at 5:21 PM, Constantinos Venetsanopoulos <[email protected] > wrote:
> [forwarding to the list too, to continue the discussion] > > > -------- Original Message -------- Subject: Re: RFC: ganeti improvements > Date: > Fri, 25 Jan 2013 18:06:57 +0200 From: Constantinos Venetsanopoulos > <[email protected]> <[email protected]> Organization: Greek Research and > Technology Network To: Guido Trotter > <[email protected]><[email protected]> > > > On 01/25/2013 03:40 PM, Guido Trotter wrote: > > On Fri, Jan 25, 2013 at 12:52 PM, Constantinos Venetsanopoulos > > <[email protected]> <[email protected]> wrote: > >> Hello all, > >> > >> > >> On 01/25/2013 01:42 PM, Guido Trotter wrote: > >>> On Fri, Jan 25, 2013 at 11:01 AM, Nicolas Sebrecht <[email protected]> > >>> <[email protected]> > >>> wrote: > >>>> [ X-post to ganeti and ganeti-devel mailing lists. ] > >>>> > >>>> > >>>> Hi all, > >>>> > >>>> Ganeti is now used in our compagny, internally. We'd like to make it > >>>> full solution ready for our customers but we are missing some critical > >>>> features. > >>>> > >>>> Before going further and starting to play with the code I'd like to > >>>> share my thoughts for comments. The idea is to submit patches in the > >>>> correct direction as early as possible in the development process. > >>>> > >>>> I didn't looked much at the code, yet. > >>>> > >>>> > >>>> 1. Improving the KVM integration. > >>>> > >>>> 1.1. Hibernation. > >>>> > >>>> AFAICT, we can't hibernate a KVM instance from Ganeti. I guess it is > >>>> possible to do with QMP and store the RAM in a dedicated path or LVM > >>>> volume. > >>>> > >>> I believe adding a gnt-instance hibernate (or shutdown --hibernate) > >>> would work well from our side. > >> > >> I agree on that. > >> > >> > >>>> 1.2. CPU Tunning. > >>>> > >>>> - CPU pinning (mask) is supported. > >>>> - CPU SMP is under work. > >>>> > >>> Yes, this is all part of 2.7. Feel free to test it further and let us > >>> know if it works. > >>> > >>>> 2. Snapshots. > >>>> > >>>> The current gnt-backup tool does not fit our needs. Here are some > >>>> limitations I've found (correct me if I'm wrong): > >>>> > >>>> - It relies on os-type (and variants?) which is fine if you have > >>>> homogeneous instances but does not help for a lot of per-instance > >>>> customing. > >>>> BTW, os-interface mix various kind of requirements (creation, > >>>> export/import, rename, verify). It is hard for us to get relevant > >>>> families of instances matching all these kind of requirements. > >>>> > >>> We do want to improve the os area, and are willing of course to > >>> discuss a design and accept patches. > >>> While allowing an OS to do something smarter is good for disk space > >>> and efficiency, we can definitely add a fallback option. > >> > >> Comments below. > >> > >> > >>>> - gnt-backup command does not support more than one export strategy. > >>>> > >>> What export strategies would you like? > >>> > >>>> - The export scripts are too much hard do understand and modify for our > >>>> system administrators. But full instances management must be > >>>> accessible for sysadmins (creation, tunning, backups, etc). > >>>> > >>> They don't seem "very" hard, but I definitely agree to any > >>> simplification, and to making sure these features can run outside the > >>> node os, if possible. (a-la-snf-image, but ganeti driven rather than > >>> os driven, perhaps?) > >> > >> Comments below. > >> > >> > >>>> - Perhaps other limitations I'm not aware? > >>>> > >>> The oses running as root on the node are a limitation if you want > >>> arbitrary users to upload their own. > >>> snf-image solves it for kvm, but it would be good to have ganeti > >>> understand the concept of "boot auxiliary image to drive os > >>> installation and other events". > >>> > >>>> I believe it might worth starting another tool from scratch with a new > >>>> design. I'm not saying it would have to replace the current > >>>> gnt-backup. I'm more suggesting something like 'gnt-snapshot' or > >>>> whatever at the side. Any thoughts on this? > >>>> > >>> I believe having gnt-backup snapshot would be as good. I don't see a > >>> particular reason for another extra command (sometimes I wonder why it > >>> should be gnt-backup at all, rather than subcommands of gnt-instance). > >> > >> Comments below. > >> > >> > >>>> 3. Clone. > >>>> > >>>> Clonning an instance looks not supported. > >>>> > >>> Marco's idea sounds good, on this. Why not gnt-backup clone that does > >>> this via export+import? (if not somehow in a smarter way?) > >> > >> The ability for clones and snapshots is a very hot issue, but from our > >> experience, > >> it's not that easy to tackle, if you want to do it correctly. I believe > >> that > >> it > >> should not be confused with the OS definitions level and the gnt-backup > >> functionality. Some food for thought: > >> > >> Currently, we are able to do full clones and snapshots using a custom > >> ExtStorage > >> provider. This provider speaks with our custom distributed storage layer > >> which > >> undertakes the actual operation. So, to create an instance by cloning an > >> existing > >> image disk known by the external system, we run: > >> > >> gnt-instance add -t ext -o snf-image+default ... > >> --disk > >> 0:size=10G,origin="disk-UUID-known-to-the-external-system" > >> > >> This way "origin" is passed to the ExtStorage provider as a parameter. The > >> provider knows from where and how to clone the disk transparently to > >> Ganeti. After that, the OS definition (snf-image) operates on a top of the > >> disk > >> and does all the customization (sets passwords, resizes the filesystem, > >> etc). > >> > >> However, we believe (and after a live talk with Iustin) that it would be > >> nice > >> to have Ganeti know about the concept of cloning and snapshotting. But to > >> do so, I believe we would need the following: > >> > >> * Disk objects should become logically detached from instances, obtain > >> their > >> own UUIDs (as has happened with networks and nodegroups) and exist > >> independently of the instances. One should be able to have disks that > >> are > >> not attached to any instance. > >> > >> * The cloning and snapshotting functionality should be implemented at the > >> bdev.py level as separate functions for every template and not at the > >> OS level. > >> > > Note that these 2 are not necessarily interdependent, and you can have > > 2 without 1. > > Of course. > > > It would be possible to start by implementing cloning and unmanaged > > snapshotting at the bdev level, and use still an instance as the > > target of cloning, and then later on do the actual separation of > > storage entities management and better management of the snapshots. > > Two questions: > > What do you mean "unmanaged snapshotting"? > > I mean having a command that "snapshots" an instance (or instance disk), but doesn't allow to do anything further with it. Another option is to allow also removing and listing, at a later stage, which starts going in the direction of (1) (but initially without for example having disks in the config file as an entity). > What do you mean "instance as the target of cloning"? > > gnt-instance create -t clone:i1 i2 So you need an instance to create an instance, and you can't just clone a block device. Another option would be going with a sort of "cloning-adoption" method. > To me, cloning has to do with disks. When you add an instance, > you should be able to specify for each disk its clone origin. > > But this can be in a later stage, or not? > >> The above would allow for very neat things apart from cloning/snapshotting, > >> such as volume attachment, detachment etc. This functionality, combined > >> with Storage Pools (comments below :) ) would open up a whole new > >> perspective to what Ganeti can do. > >> > > ACK, what do you think of the plan of doing it in two stages, as outlined > > above? > > (to have the functionality earlier and also not to make it depend on > > something quite big, but actually vice versa to build the detachment > > on the grounds of it) > > > > That was exactly what I would propose too. First implement cloning > and snapshotting in bdev.py, find a way to define them at the command > line per disk (new IDISK_PARAMS?) and then go on with the design > change of disk separation, which will lead to the need of disk management. > And along with that management of snapshots. > > It could be done in a similar way to adoption, I believe, perhaps. > >> > >>>> 4. Different disk types for an instance. > >>>> > >>>> I've heard some discussions about that but can't remember what's the > >>>> current status. Not urgent at all but I guess this is something we will > >>>> need some day. > >>>> > >>> There is a design for storage pools, but it's not being implemented > >>> yet. We definitely would like to have it in. > >> > >> Storage Pools are coming :) After merging the ExtStorage patch series, > >> this is the next to come. During the ExtStorage merge we had some > >> interesting discussions with Iustin that have to do with Storage Pools, > >> so I'll try to wrap everything up and revive the discussion on their > >> design as soon as possible. > >> > > ACK, thanks a lot for your help on this. > > No problem. Will come back on it as soon as possible. > > Thanks, but you're right, let's merge hotplug first. Guido --
