On 15 Jun 2015, at 17:57, Nir Soffer <nsof...@redhat.com> wrote:
> ----- Original Message ----- >> From: "Michal Skrivanek" <mskri...@redhat.com> >> To: "Nir Soffer" <nsof...@redhat.com> >> Cc: "Francesco Romani" <from...@redhat.com>, "devel" <devel@ovirt.org>, >> "Adam Litke" <ali...@redhat.com>, "Federico >> Simoncelli" <fsimo...@redhat.com>, "Dan Kenigsberg" <dan...@redhat.com>, >> "Allon Mureinik" <amure...@redhat.com>, >> "Daniel Erez" <de...@redhat.com>, "Eric Blake" <ebl...@redhat.com> >> Sent: Monday, June 15, 2015 6:20:34 PM >> Subject: Re: Libvirt secrets management - take 2 >> >> >> On 15 Jun 2015, at 17:06, Nir Soffer wrote: >> >>> >>> >>> ----- Original Message ----- >>>> From: "Francesco Romani" <from...@redhat.com> >>>> To: "devel" <devel@ovirt.org> >>>> Cc: "Adam Litke" <ali...@redhat.com>, "Federico Simoncelli" >>>> <fsimo...@redhat.com>, "Dan Kenigsberg" >>>> <dan...@redhat.com>, "Allon Mureinik" <amure...@redhat.com>, "Daniel Erez" >>>> <de...@redhat.com>, "Michal Skrivanek" >>>> <mskri...@redhat.com>, "Eric Blake" <ebl...@redhat.com>, "Nir Soffer" >>>> <nsof...@redhat.com> >>>> Sent: Monday, June 15, 2015 4:08:09 PM >>>> Subject: Re: Libvirt secrets management - take 2 >>>> >>>> ----- Original Message ----- >>>>> From: "Nir Soffer" <nsof...@redhat.com> >>>>> To: "Adam Litke" <ali...@redhat.com> >>>>> Cc: "devel" <devel@ovirt.org>, "Francesco Romani" <from...@redhat.com>, >>>>> "Federico Simoncelli" <fsimo...@redhat.com>, >>>>> "Dan Kenigsberg" <dan...@redhat.com>, "Allon Mureinik" >>>>> <amure...@redhat.com>, "Daniel Erez" <de...@redhat.com>, >>>>> "Michal Skrivanek" <mskri...@redhat.com>, "Eric Blake" >>>>> <ebl...@redhat.com> >>>>> Sent: Saturday, June 13, 2015 11:14:03 PM >>>>> Subject: Re: Libvirt secrets management - take 2 >>>> >>>> [...] >>>>>>> Flows >>>>>>> ===== >>>>>>> >>>>>>> Start vm >>>>>>> -------- >>>>>>> - Engine add required secrets to vm description >>>>>>> - Vdsm register temporary secrets with libvirt >>>>>>> - When vm is up or if operation failed, Vdsm remove the temporary >>>>>>> secret >>>>>>> >>>>>>> Migrate vm >>>>>>> ---------- >>>>>>> - Engine add required secrets to vm description >>>>>>> - Vdsm add secrets to the vm description sent to the destination >>>>>>> - On the destination, Vdsm register temporary secrets with libvirt >>>>>>> - On the destination, when vm is up or if operation failed, Vdsm remove >>>>>>> the temporary secret >>>>> >>>>> On issue with migration - do we have to keep the same auth information as >>>>> in >>>>> the original vm xml, or we can create the vm on the destination using >>>>> different >>>>> auth xml? >>>> >>>> I don't have an answer for this. Need to check and play with this a bit. >>>> >>>>>>> Hot-plug disk >>>>>>> ------------- >>>>>>> - Engine add secret to disk description >>>>>>> - Vdsm register temporary secret with libvirt >>>>>>> - When disk is successfully plugged, or if operation failed, Vdsm >>>>>>> remove >>>>>>> the temporary secret >>>> >>>> OK, what if this VM is migrate afterwards? destination VDSM needs to >>>> register >>>> this secret >>>> to libvirt. >>>> >>>> The fact here is that is the source VDSM that orchestrates the migration, >>>> and >>>> if it has >>>> forgorgotten the secret, then we need get it back again from Engine and >>>> pass >>>> it through the destination VM. >>>> >>>> On migration, it is the source VM which orchestrates the flow. Engine >>>> polls >>>> source VM for updates, >>>> and consider the VM migrated only after source exited correctly. >>>> I don't recall the fine details engine side (need to check), but on VDSM >>>> side: >>>> >>>> - source VDSM receives VM.migrate(destination_uri) >>>> - source VDSM sends migrationCreate to destination VDSM >>>> - once destination VDSM created special-purpose VM, source VDSM instructs >>>> libvirt to start the migration >>>> - source VDSM monitors the migration, report progress, updates the >>>> downtime >>>> - migration fails: source VDSM automatically send VM.destroy to >>>> destination >>>> VDSM >>>> - migration succeeds: source VDSM destroy source VM, execution now >>>> continues >>>> on destination VM >> >> the source side is "destroyed" by qemu automatically. Engine just sends an >> explicit destroy after the migration is deemed successful in the engine too >> >>> >>> Ok, when we create the special purpose vm, do we include all the devices? >> >> it's an exact copy of the source vm. yes >> >>> >>> If this special vm (waiting for migration) is connected to all the disks, >>> then >> >> it is (it goes though preparePaths as any other VM creation) >> >>> we only need to register the secrets sent from the source in >>> migrationCreate, >>> start the special vm, and one it is running, delete the secrets. >> >> probably. Does the source know those secrets? > > At the time of migration, the source does not know much about the secrerts. > > - qemu got the secret key and connected to the ceph disks. > - libvirt knew about the secret when starting qemu/ hot-plugging a disk, but > now it does not know anything about the secret, except the secret uuid > in the <disk> <auth> element. So when the VM loses ceph connection and needs to reconnect is the uuid enough? > - vdsm knows about the auth dicts in the disk devices, but does not know the > key (the key cannot be persisted) Assuming it can reconnect (as above) using the auth, can't the new destination VM use the same? > > So engine will have to send the secrets again, so vdsm can send them to the > destination. Doable. But the the vdsm can cache it as well, perhaps exluded from saveState(),but otherwise why not? Unrelated Q - AFIK it's supposed to work on ppc as well. Current ceph-common have no ppc64le builds. When is this going to be added? Thanks, michal > >> >>> >>>> >>>> so far, Engine talks with destination VM only if migration completed >>>> succesfully (unless I'm missing something :)) >>>> >>>> so we need to pass back again all the needed secrets as part of VM.migrate >>>> verb >>>> >>>> This is only the very first thought of mine, we can probably think >>>> something >>>> cleaner. >>>> >>>> Bests, >>>> >>>> -- >>>> Francesco Romani >>>> RedHat Engineering Virtualization R & D >>>> Phone: 8261328 >>>> IRC: fromani >> >> _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
