Re: [Engine-devel] Domain rescan action question
On 08/02/2012 04:44 AM, Shu Ming wrote: No comments below. I am just curious how you can have disk created outside of the oVirt environment if the storage domain is mounted on oVirt engine. Does your application try to communicate with VDSM though SPM host to do that? Are the two applications(oVirt engine and your application) coexisting and manipulating the storage domain at the same time or one application is quiet when the other one is functioning? IMHO, modifying the storage domain meta data from two application are dangerous and I suppose your application should know the reason and what these disks are for. So telling oVirt to import the disks automatically should be reasonable because you know what these disks are for and it is unnecessary for the oVirt administrator to select which one to import. two use cases: 1. NFS storage allows this quite easily, and very relevant with storage side cloning. 2. engine and storage go out of sync (recover either from backup) On 2012-8-2 1:36, Hopper, Ricky wrote: On 8/1/12 10:05 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 4:56:45 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. I think are nice to have, but the problem with the scanning is that if we're not scanning a master domain or an export domain, all we will see is a bunch of images with no context or even hints as to where they belong. The data that makes it all usable is in the engine database and in the ovf files on the master domain. This is why I stopped at the orphaned images part of the feature - because there it's feasible, I would
Re: [Engine-devel] Domain rescan action question
In the interest of good discussion, we've put up a feature page for this feature (http://wiki.ovirt.org/wiki/Features/Domain_Scan), which links to a talk page where modifications can be proposed to how I've laid out the feature. So far, it covers how the query works and which commands will come about to implement it. I'd appreciate it if anyone concerned could check this out and make any changes as they see fit so we can get going with the coding. - Ricky On 8/1/12 2:35 PM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 8:36:09 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 10:05 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 4:56:45 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. I think are nice to have, but the problem with the scanning
Re: [Engine-devel] Domain rescan action question
Thanks Ricky, I've added reviewing this to my todo list Dan - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org Sent: Thursday, 2 August, 2012 5:37:10 PM Subject: Re: [Engine-devel] Domain rescan action question In the interest of good discussion, we've put up a feature page for this feature (http://wiki.ovirt.org/wiki/Features/Domain_Scan), which links to a talk page where modifications can be proposed to how I've laid out the feature. So far, it covers how the query works and which commands will come about to implement it. I'd appreciate it if anyone concerned could check this out and make any changes as they see fit so we can get going with the coding. - Ricky On 8/1/12 2:35 PM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 8:36:09 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 10:05 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 4:56:45 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned
Re: [Engine-devel] Domain rescan action question
- Original Message -
On 08/01/2012 12:21 AM, Ayal Baron wrote:
- Original Message -
On 07/31/2012 11:30 PM, Hopper, Ricky wrote:
Hey all,
As I'm making progress with the domain rescan functionality, I've
realized that I'm unsure what to do with any disks that are
detected on
the domain. Should I add them back into the database to be listed
as
floating disks, or should I just return a list of disk images to
be
attached to whatever the caller of the query needs?
- Ricky
i'm not sure they should be added automatically.
I think a dialog[1] showing orphan disks/images on the storage
domain
for user to choose which to import as 'floating' disks would be
better
than auto importing them.
why? this same functionality would be used to import an existing
domain.
If these disks are referenced in OVFs we are not familiar with on
this domain then we should import the *VMs*.
If they are referenced by other VMs that are already in the system
(but disks have been unreachable until now) then the disks should
just be added to the db in attached mode.
If neither, then the disks should be added as floating disks.
For the import functionality, once you subsequently import another
domain with OVFs which do reference these disks then if user
hasn't appropriated them for other VMs then they would move to
attached state, otherwise need to add those VMs with errors.
Note that there are 2 reasons for unknown valid disks to be on the
domain:
1. delete was initiated in engine but not performed on storage
(then floating is fine as only way to automatically delete them is
if user chooses to do so)
2. disks were created there outside of the system - should just
detect and import and use logic above.
there is also the reverse of flagging existing disks as 'missing'
in
storage?
If disks were floating then they should just be removed, otherwise
should be moved to illegal state (we have this state for disks
today).
[1] or a subtab on the storage domain.
Another sub-tab for disks?
It's possible but what would you do when importing an existing
domain into the system? require user to manually select which
disks to import? or would you have different flows for import of
domain and rescan of contents? (I'd rather keep it simple with
less tabs and manual operations for user to perform, seems more
intuitive to me).
I'm not sure orphaed images and importing an entire domain should be
the
same flow from UX part.
and for a netapp native clone, you'd want to only import the newly
cloned (orphaned) image from the storage via an api call, not the
entire domain/images
For proper bookkeeping I think we'd want these disks to be reflected in the
GUI. I do not however think we need to add additional tabs for this.
We could have yet another state ('orphaned' or sth.) for disks to be able to
easily differentiate in the GUI etc.
I would imagine we'd regularly scan domains to account for storage usage.
If we do add such a state then we'd need an API to 'enable' it (so that it
could be done automatically).
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
- Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, August 1, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images +1 also, how will be handled images with snapshots? or broken chain of images (not sure its a valid scenario) [1] or a subtab on the storage domain. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
- Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? [1] or a subtab on the storage domain. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- Regards, Dan Yasny Red Hat Israel +972 9769 2280 ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. [1] or a subtab on the storage domain. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- Regards, Dan Yasny Red Hat Israel +972 9769 2280 ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. I think are nice to have, but the problem with the scanning is that if we're not scanning a master domain or an export domain, all we will see is a bunch of images with no context or even hints as to where they belong. The data that makes it all usable is in the engine database and in the ovf files on the master domain. This is why I stopped at the orphaned images part of the feature - because there it's feasible, I would rely on the engine database for image ID comparisons. If we present a user with a list of nameless disks, I doubt it will be of any use. The way this would work is by comparing a list of disk images from vdsm and from oVirt's database, finding the ones vdsm returns that oVirt doesn't have, and then either adding or returning those images. So oVirt's db will be used in the comparison. As far as presenting the user with nameless disks, that's a point I hadn't considered; we could generate some sort of placeholder metadata upon addition to show the user that these are new/orphaned disks that were found on the storage domain. Is it safe to assume that the disks discovered by this feature won't be attached to anything? [1] or a subtab on the storage domain. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel -- Regards, Dan Yasny Red Hat Israel +972 9769 2280 -- Regards, Dan Yasny Red Hat Israel +972 9769 2280 ___ Engine-devel mailing list Engine-devel@ovirt.org http
Re: [Engine-devel] Domain rescan action question
- Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 4:56:45 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. I think are nice to have, but the problem with the scanning is that if we're not scanning a master domain or an export domain, all we will see is a bunch of images with no context or even hints as to where they belong. The data that makes it all usable is in the engine database and in the ovf files on the master domain. This is why I stopped at the orphaned images part of the feature - because there it's feasible, I would rely on the engine database for image ID comparisons. If we present a user with a list of nameless disks, I doubt it will be of any use. The way this would work is by comparing a list of disk images from vdsm and from oVirt's database, finding the ones vdsm returns that oVirt doesn't have, and then either adding or returning those images. So oVirt's db will be used in the comparison. This will work only when scanning storage domains already attached and in use by the current oVirt setup. What I am talking about is what will happen if a LUN that used to be a SD in another oVirt setup is discovered and scanned, with no engine db to compare with. If we don't consider such a use case, life
Re: [Engine-devel] Domain rescan action question
On 8/1/12 10:05 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 4:56:45 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. I think are nice to have, but the problem with the scanning is that if we're not scanning a master domain or an export domain, all we will see is a bunch of images with no context or even hints as to where they belong. The data that makes it all usable is in the engine database and in the ovf files on the master domain. This is why I stopped at the orphaned images part of the feature - because there it's feasible, I would rely on the engine database for image ID comparisons. If we present a user with a list of nameless disks, I doubt it will be of any use. The way this would work is by comparing a list of disk images from vdsm and from oVirt's database, finding the ones vdsm returns that oVirt doesn't have, and then either adding or returning those images. So oVirt's db will be used in the comparison. This will work only when scanning storage domains already attached and in use by the current oVirt setup. What I am talking about is what will happen if a LUN that used to be a SD in another oVirt setup is discovered and scanned, with no engine db
Re: [Engine-devel] Domain rescan action question
No comments below. I am just curious how you can have disk created outside of the oVirt environment if the storage domain is mounted on oVirt engine. Does your application try to communicate with VDSM though SPM host to do that? Are the two applications(oVirt engine and your application) coexisting and manipulating the storage domain at the same time or one application is quiet when the other one is functioning? IMHO, modifying the storage domain meta data from two application are dangerous and I suppose your application should know the reason and what these disks are for. So telling oVirt to import the disks automatically should be reasonable because you know what these disks are for and it is unnecessary for the oVirt administrator to select which one to import. On 2012-8-2 1:36, Hopper, Ricky wrote: On 8/1/12 10:05 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Andrew Cathrow acath...@redhat.com Sent: Wednesday, 1 August, 2012 4:56:45 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 9:42 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Ricky Hopper ricky.hop...@netapp.com To: Dan Yasny dya...@redhat.com, Andrew Cathrow acath...@redhat.com Cc: engine-devel@ovirt.org, Itamar Heim ih...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Sent: Wednesday, 1 August, 2012 4:34:53 PM Subject: Re: [Engine-devel] Domain rescan action question On 8/1/12 5:59 AM, Dan Yasny dya...@redhat.com wrote: - Original Message - From: Andrew Cathrow acath...@redhat.com To: Itamar Heim ih...@redhat.com, Dan Yasny dya...@redhat.com, Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Wednesday, 1 August, 2012 12:24:42 AM Subject: Re: [Engine-devel] Domain rescan action question - Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images The original idea was to scan the storage domains and compare the images lists to the database, thus getting a list of images no longer relevant and scrubbing the storage. This will actually be addressed properly in the future (Ayal can elaborate on that) but for now this is needed at least for that use case. As I understand, the conversation here is about trying to take an already populated SD (from another setup I suppose), scanning it and putting it into RHEV? As I understood it, the purpose of this functionality wasn't to find images which should be removed from storage, but to find images on the domain that oVirt was unaware of and importing them for use (for instance, if a disk was created outside of oVirt on the domain). If one of the use cases for this feature is also the orphaned images mentioned on the feature page, that may expand the functionality into a separate domain scrub and storage import, both of which would call the rescan (meaning the rescan would not actually add to the database, but instead return a list of orphaned disk images). Another solution would be to import all disk images into the database either way, and let the user delete any orphaned images from the GUI. I think are nice to have, but the problem with the scanning is that if we're not scanning a master domain or an export domain, all we will see is a bunch of images with no context or even hints as to where they belong. The data that makes it all usable is in the engine database and in the ovf files on the master domain. This is why I stopped at the orphaned images part of the feature - because there it's feasible, I would rely on the engine database for image ID comparisons. If we present a user with a list of nameless disks, I doubt it will be of any use. The way this would work is by comparing a list of disk images from
Re: [Engine-devel] Domain rescan action question
On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? [1] or a subtab on the storage domain. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
- Original Message - On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. why? this same functionality would be used to import an existing domain. If these disks are referenced in OVFs we are not familiar with on this domain then we should import the *VMs*. If they are referenced by other VMs that are already in the system (but disks have been unreachable until now) then the disks should just be added to the db in attached mode. If neither, then the disks should be added as floating disks. For the import functionality, once you subsequently import another domain with OVFs which do reference these disks then if user hasn't appropriated them for other VMs then they would move to attached state, otherwise need to add those VMs with errors. Note that there are 2 reasons for unknown valid disks to be on the domain: 1. delete was initiated in engine but not performed on storage (then floating is fine as only way to automatically delete them is if user chooses to do so) 2. disks were created there outside of the system - should just detect and import and use logic above. there is also the reverse of flagging existing disks as 'missing' in storage? If disks were floating then they should just be removed, otherwise should be moved to illegal state (we have this state for disks today). [1] or a subtab on the storage domain. Another sub-tab for disks? It's possible but what would you do when importing an existing domain into the system? require user to manually select which disks to import? or would you have different flows for import of domain and rescan of contents? (I'd rather keep it simple with less tabs and manual operations for user to perform, seems more intuitive to me). ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
- Original Message - From: Itamar Heim ih...@redhat.com To: Ricky Hopper ricky.hop...@netapp.com Cc: engine-devel@ovirt.org Sent: Tuesday, July 31, 2012 4:44:34 PM Subject: Re: [Engine-devel] Domain rescan action question On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. there is also the reverse of flagging existing disks as 'missing' in storage? Perhaps we should start a feature page to discuss and better scope it. There is a feature page that we could expand, it doesn't discuss the notion of importing those disks which is certainly something we need to address. http://wiki.ovirt.org/wiki/Features/Orphaned_Images [1] or a subtab on the storage domain. ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Re: [Engine-devel] Domain rescan action question
On 08/01/2012 12:21 AM, Ayal Baron wrote: - Original Message - On 07/31/2012 11:30 PM, Hopper, Ricky wrote: Hey all, As I'm making progress with the domain rescan functionality, I've realized that I'm unsure what to do with any disks that are detected on the domain. Should I add them back into the database to be listed as floating disks, or should I just return a list of disk images to be attached to whatever the caller of the query needs? - Ricky i'm not sure they should be added automatically. I think a dialog[1] showing orphan disks/images on the storage domain for user to choose which to import as 'floating' disks would be better than auto importing them. why? this same functionality would be used to import an existing domain. If these disks are referenced in OVFs we are not familiar with on this domain then we should import the *VMs*. If they are referenced by other VMs that are already in the system (but disks have been unreachable until now) then the disks should just be added to the db in attached mode. If neither, then the disks should be added as floating disks. For the import functionality, once you subsequently import another domain with OVFs which do reference these disks then if user hasn't appropriated them for other VMs then they would move to attached state, otherwise need to add those VMs with errors. Note that there are 2 reasons for unknown valid disks to be on the domain: 1. delete was initiated in engine but not performed on storage (then floating is fine as only way to automatically delete them is if user chooses to do so) 2. disks were created there outside of the system - should just detect and import and use logic above. there is also the reverse of flagging existing disks as 'missing' in storage? If disks were floating then they should just be removed, otherwise should be moved to illegal state (we have this state for disks today). [1] or a subtab on the storage domain. Another sub-tab for disks? It's possible but what would you do when importing an existing domain into the system? require user to manually select which disks to import? or would you have different flows for import of domain and rescan of contents? (I'd rather keep it simple with less tabs and manual operations for user to perform, seems more intuitive to me). I'm not sure orphaed images and importing an entire domain should be the same flow from UX part. and for a netapp native clone, you'd want to only import the newly cloned (orphaned) image from the storage via an api call, not the entire domain/images ___ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
