Given a couple of questions I have gotten in response to my survey responses... let me clarify, although I imply we do a significant volume of V2V, which we do as well, the responses below are targeted and applicable to cloning. Especially the need to re-personalize clones beyond the OS scope to the application scope when and where possible.
We maintain the original VMs in the original environment, often running clones in parallel on different environments, so cloning is done, both within the same hypervisor scope and across different hypervisor environments. Our work with SPECVirt is one concrete example where we use cloning routinely, to build multiple tiers or tiles. Our focus thus far has been in the lab/test/development space, but our client base is pushing for same flexibility in QA/production environments, so the need for better/mature tools and methods in the KVM space, including cloning, is key to our future goals. Schorschi Decker VP; Sr. Consultant Engineer ET&D Emerging Technologies / Virtualization Platform Engineering Team Bank of America -----Original Message----- From: Decker, Schorschi Sent: Tuesday, 10 May, 2011 07:01 To: virt-tools-l...@redhat.com; Cole Robinson Cc: v...@lists.fedoraproject.org; qemu-devel@nongnu.org Subject: RE: [virt-tools-list] Virt Tools Survey: What to do about virt-clone (a) Is cloning guests useful for you or not? Often or infrequently? Our end-users are quite virtualization aware, they have demanded that we support a number of features in our virtual infrastructure, this included. It will be used frequently and often, maybe 100s of times a day across our global infrastructure. (b) Do you currently use virt-clone to clone guests? Only in the lab, cloning or templating is a new initiative in our maturing design, so the timing of revamping this is perfect for our plans over the next few months. (c) Do you have a homebrew method to clone guests? What does it do? Yes, we often migrate and/or clone VMs in the lab between VMware and KVM and Hyper-V and back again. VMware and KVM is quite easy, given the flexibility of common shared storage support and VM disks in RAW format adaptable to VMware and KVM. Hyper-V has its own challenges given its limitations by design. For VMware to KVM and back, we have a few bash scripts that we use. Migrations are straight-forward, since no personality customization is needed, but true clones, which we use for SPECVirt analysis for example, we have to remove/reapply personality. This is not trivial, since we have a lot of applications in our base image that are infrastructure aware, and require reconfiguration, or re-initialization, beyond the typical sysprep or OS level changes. Windows is especially painful on this point. Our personality management scripting is in a constant state of change because we are routinely updating our core images. (d) Do you use another tool to clone guests? (And how is it?) We have used platespin off and on, and of course use various tools and methods core to WAIK tool set for Windows. But custom scripting is the significant 800 pound gorilla in the room on this point. A specific in VM agent or service that would explicitly support re-personalization regardless of OS or application level would be of great benefit, since meta-data or a back-end database could drive the re-personalization, provided a more stateless model. Especially for Windows this might be an idea worth consideration. BladeLogic, Ops-ware, Altaris, etc. all use some form of this idea, but they are also focused on provisioning in general so the meta-data driven re-personalization gets lost in the overall product. (e) When you clone a guest, do you "sysprep" it or would you like to? (Using the term "sysprep" generically here, I mean any sort of reinitialization for Linux or Windows guests). Sysprep is fine as a place to start, but even when Microsoft wrote sysprep, it was an idea that was ignored for a long time, and it was never focused on, the world has since gone forward, so sysprep is at best, only a starting point for what is needed to abstract personality. A better solution would be that we image/clone the personality specific deltas and archive them to a library, rather than complete clones that have personality logic staged for implementation on first boot. (f) How do you feel about a multi-step process? virt-clone -> virt-sysprep -> virt-resize (for example) As long as any GUI and CLI (as well as API) are consistent in feature set and flow, multi-step is fine. We need to be able to completely automate the process, since we would drive it from a self-serve web portal concept. (g) Have you had other problems with cloning guests? Nothing beyond what was outlined above. Abstracting of personality at OS and application level is key for us. (h) What have I missed out in this analysis? What other features have you missed in virt-clone? No specifically to virt-clone but in general, abstracting the common code, to common library or binary, so that virt-manager, virsh, etc. all reference the same design, feature, and executables, is the strategic direction that should be followed. The idea that different code is trying to implement same concepts in different tools makes no long term strategic sense for the KVM platform in general. Schorschi Decker VP; Sr. Consultant Engineer ET&D Emerging Technologies / Virtualization Platform Engineering Team Bank of America -----Original Message----- From: virt-tools-list-boun...@redhat.com [mailto:virt-tools-list-boun...@redhat.com] On Behalf Of Richard W.M. Jones Sent: Tuesday, 10 May, 2011 04:57 To: virt-tools-l...@redhat.com; Cole Robinson Cc: v...@lists.fedoraproject.org; qemu-devel@nongnu.org Subject: [virt-tools-list] Virt Tools Survey: What to do about virt-clone I've volunteered for the task of fixing virt-clone[0]. There are a number of bugs which need to be addressed. Unfortunately the current virt-clone is broken-by-design since it cannot make changes inside the guest. [0] http://linux.die.net/man/1/virt-clone The bugs boil down to what Microsoft calls "sysprepping" the clone, which is to say, removing its existing identity, hostname, ssh host keys, persistent network rules, host SID and workgroup name (for Windows). It's helpful for Linux guests to remove some of this stuff[1] -- it will make the cloning process smoother. For Windows it's absolutely required[2]. [1] https://rwmj.wordpress.com/2010/09/24/tip-my-procedure-for-cloning-a-fed ora-vm/ [2] http://technet.microsoft.com/en-us/library/cc721940%28WS.10%29.aspx All that virt-clone can do now is to copy the guest and make some simple changes to the libvirt XML (eg. giving it a new MAC address). It doesn't even address the sysprepping problem. The problem with sysprepping is that it's hard to do, and it's different for every operating system. I've summarized some of the techniques below. Worse than that, for some OSes there are different levels of sysprepping that an administrator might want; also see below. So I'd like feedback from "virt-clone next generation" users: (a) Is cloning guests useful for you or not? Often or infrequently? (b) Do you currently use virt-clone to clone guests? (c) Do you have a homebrew method to clone guests? What does it do? (d) Do you use another tool to clone guests? (And how is it?) (e) When you clone a guest, do you "sysprep" it or would you like to? (Using the term "sysprep" generically here, I mean any sort of reinitialization for Linux or Windows guests). (f) How do you feel about a multi-step process? virt-clone -> virt-sysprep -> virt-resize (for example) (g) Have you had other problems with cloning guests? (h) What have I missed out in this analysis? What other features have you missed in virt-clone? Sysprepping Windows ------------------- This is a complex, manual process. We do some steps to automate it in RHEV. It's best to read Microsoft's online documentation at [2][3][4]. [3] http://support.microsoft.com/kb/302577 [4] http://blogs.technet.com/b/megand/archive/2005/01/20/357570.aspx Fedora ------ In theory you can just write a file /.unconfigured in the root, and Fedora will go through the firstboot process at next boot (it will reset timezone, root password, netconfig, keyboard, authentication). Some admins will *not* want all of these things to be reset, and will want either a lesser degree of unconfiguration, or will want to control each thing manually. I'm not totally convinced that this hasn't been broken by systemd introduction in Fedora 15. general Linux ------------- See [1]. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/ _______________________________________________ virt-tools-list mailing list virt-tools-l...@redhat.com https://www.redhat.com/mailman/listinfo/virt-tools-list ---------------------------------------------------------------------- This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing.