Hi Marios,
Thanks for starting to look at the patch and issues!
> * Image description isn't being captured - could well be that this
> doesn't exist but I noticed :description => img['description'][0].to_s
> in 'def images' so i note it here in case
The currently available images do not have descriptions. I have created
an image myself with description and it was listed properly in DC's GUI.
> * Portal *does* work for google chrome. It was helpful to match the
> portal view with what deltacloud was showing me (e.g. for realms) to
> start to understand the fgcp concepts.
Glad to hear that. Means you can also download the manuals from the
portal :)
> * I couldn't launch an instance. Initially because I didn't understand
> the difference between realms 'vsys' and the others. But then using
A good reason to try to somehow separate the realms by scope!
> I get: Error: User doesn't have the right of access.
[Org-ID:UZXC0GRT,
> User-ID:marios, Resource-ID:UZXC0GRT-9Q988189J, Action-
> ID:ResourceExec]
Glad to see the error message is properly propagated :)
As I mentioned on IRC, I had created that vsys for my testing, not
thinking it would be around long enough that you'd try to use it.
By default a new vsys can only be operated by the person who created it.
I have updated your and David's privileges so that you can also operate
that vsys now. So you can use the one I created (and don't worry about
accidentally deleting anything, it's just there for testing and easy to
recreate if I need it again), but you're also free to create your own.
> *[for later but]... could tidy up what the client is outputting to the
> console (all the http requests/responses for example) - could consider
> adding a 'debug mode' for this - but definitely not the most important
> thing right now.
Yes, let's safe that change for last :)
Does DC already have a debug mode option that I can use?
> * I like the solution you came up with for the authentication. Needs
to
> be well documented though (in due course)... e.g. that apart from the
> env variables, credentials.name has to be the subdirectory for the
user
> cert, and credentials.password is the cert pass.
Thanks. David suggested we could consider adding a certificate upload
API later. I believe the current solution is not too restrictive for
now.
> > -> Suggestion: Make the image id variable optional. Additionally, we
[...]
> creation status/progress. For now, "PENDING#{opts[:name]}" for example
> is the best we could do, and document this behaviour...
OK.
The reason I suggested to omit the value is that a multi-cloud client
app could be written to understand there is no id (yet), while with
"PENDING#{opts[:name]}" you're returning a string that it may interpret
as a permanent url to the new image.
At least it would be nice if we could change the DC common code not to
create a hyperlink for this type of id?
> This could work - I can't think of a way using the current API
entities
> (e.g. Provider --> Vsys won't work). However this is a big issue and
I'd
> advise starting a separate thread for this one - an initial proposal
> would be helpful (e.g. you mention a 'type' attribute above)
I have a separate thread open on that (see 'using realms for vsys
concept (FGCP driver implementation issue)' started on 9 March 2012) but
not much response.
Cheers,
Dies Koper
> -----Original Message-----
> From: [email protected] [mailto:[email protected]]
> Sent: Thursday, 3 May 2012 8:28 PM
> To: [email protected]
> Cc: Koper, Dies
> Subject: Re: outstanding issues with FGCP driver
>
> Hi Dies:
>
> prelim. notes:
>
> * Image description isn't being captured - could well be that this
> doesn't exist but I noticed :description => img['description'][0].to_s
> in 'def images' so i note it here in case
>
> * Portal *does* work for google chrome. It was helpful to match the
> portal view with what deltacloud was showing me (e.g. for realms) to
> start to understand the fgcp concepts.
>
> * I couldn't launch an instance. Initially because I didn't understand
> the difference between realms 'vsys' and the others. But then using
>
> image: /api/images/IMG_3c9820_J55YW6UE101DS
> realm: Dies-DeltaCloud UZXC0GRT-9Q988189J-N-DMZ, [Network segment]
>
> I get: Error: User doesn't have the right of access.
[Org-ID:UZXC0GRT,
> User-ID:marios, Resource-ID:UZXC0GRT-9Q988189J, Action-
> ID:ResourceExec]
>
> *[for later but]... could tidy up what the client is outputting to the
> console (all the http requests/responses for example) - could consider
> adding a 'debug mode' for this - but definitely not the most important
> thing right now.
>
> * I like the solution you came up with for the authentication. Needs
to
> be well documented though (in due course)... e.g. that apart from the
> env variables, credentials.name has to be the subdirectory for the
user
> cert, and credentials.password is the cert pass.
>
> Your list of issues below is huge (!), and will take a while to get
> addressed/commented on but got to start somewhere so:
>
> On 30/04/12 16:45, Koper, Dies wrote:
>
> 8<---snip---8<
> >
> > 1. FGCP API's image creation API does not return ID straight away
> >
> > I mentioned this before, FGCP's image creation API does not return
the
> > image id straight away (same for public IPs and storage snapshots,
> > covered below). Once the image creation has completed, it will be
> > returned by the image list API, including the new id. The FGCP
Portal
> > has an Image Manager screen which shows the image creation progress
> but
> > there is no API for it.
> > -> Suggestion: Make the image id variable optional. Additionally, we
>
> I don't see how this would help though. It would only allow you to
ommit
> 'pending' from the Image you return from 'create_image' but still
> wouldn't give the user indication of the status. An immediate thought
is
> to consider adding some kind of 'Job' entity to deltacloud but even
this
> wouldn't be much use, since FGCP doesn't provide API access to the
image
> creation status/progress. For now, "PENDING#{opts[:name]}" for example
> is the best we could do, and document this behaviour...
>
> > could consider implementing images(opts[:name]) so the user does
have
> a
> > fairly unique url to poll for it. (The FGCP driver would still need
to
> > retrieve the whole image list so it's more of a convenience method
than
> > a performance improvement.)
> >
> > 2. Realms mapping
> >
> > I'm currently using the following mapping, which helped me map all
> > relevant operations:
> >
> > DC FGCP Used by collection
> > Realm network segment Instances, load balancers
> > Realm VSYS Storage, public IPs,
image
> > creation
> > Firewall VSYS Firewall creation
> >
> > The only issue with mapping two FGCP concepts to Realms is that when
> > creating an instance from an image through the UI, it displays both
> > network segment realms and VSYS realms, but only the former work.
> > Similarly, when creating a new volume or IP address, both are
displayed
> > (but I made the driver accept both as I can determine the VSYS from
the
> > network segment).
> >
> > -> Suggestion: Introduce parent-child hierarchy (sub-realms) and
list
> > them in a tree structure in the UI. Also introduce a
> > type/scope/purpose(/limit?) that can be filtered on so that the
> > instance, IP address and storage creation operations can list only
the
> > realms that are applicable.
>
> This could work - I can't think of a way using the current API
entities
> (e.g. Provider --> Vsys won't work). However this is a big issue and
I'd
> advise starting a separate thread for this one - an initial proposal
> would be helpful (e.g. you mention a 'type' attribute above)
>
> >
> > 3. Instances details
> >
> > The FGCP driver needs to make multiple backend calls to retrieve all
> > instance details per instance and can't complete before the browser
> > times out when the number of instances grows. My driver therefore
> > retrieves only minimal detail for the instances operation and full
> > detail for the instance (or instances with :id specified) operation.
> > How can a client determine whether the instances operation has
returned
> > all available details or whether it's worth invoking instance() to
> > possibly get more details?
>
> I think this is a good compromise. We do something similar with
Buckets
> collection in that GET single bucket will give you details about the
> Blobs it contains, but GET index gives you only name and size. I think
> we could just document this in the API (i.e. that index operations may
> not return all details of the given resource)
>
>
> >
> > 4. Instance creation in realm
> >
> > As mentioned in 2., would be nice if only the relevant realms (i.e.
the
> > network segments) where listed.
> >
> > 5. Instance hwp change
> >
> > The FGCP, and I suppose other clouds, allow a user to change the
> > hardware profile of an instance (e.g. increasing memory/cpu) when
the
> > instance is not running. I believe DC does not support this yet.
Have
> > you had no requests for that yet?
> >
>
> haven't heard of any requests yet. Looking at the current API, POST
> /api/instances creates an instance, POST /api/instances/:id/:action
> invokes a start/stop/etc action. We *could* consider adding a POST
> /api/instances/:id/:update action with POST body for the updated hwp
>
>
> I leave it there for now as this is already long enough and should be
a
> start for the next iteration of driver/proposals etc,
>
> marios
>
> > 6. Storage volumes in realm
> >
> > As mentioned in 2., would be nice if only the relevant realms (i.e.
> > VSYS) where listed. Instead of only the first one as the UI does.
> >
> > 7. Storage volume types and snapshots
> >
> > FGCP API returns two types of volumes, system volumes (which are the
> > volumes instances boot from, created from the disk images) and
> > additional storage (additional, detachable volumes). The latter are
> > always empty at creation and cannot currently be cloned.
> > Both types of volume can be backed up and restored. Restoring means
> > overwriting whatever you have on it now, no flexibility to use it
for
> > cloning.
> > My driver's storage_volumes operation is returning both. What does
the
> > undocumented ':kind' attribute mean? Shall I use it to specify the
type
> > of storage (system|additional)?
> >
> > 8. Storage volume attached
> >
> > Even if my driver specifies that the volume is not attached (i.e.
> > :instance_id => nil), the UI says it is attached to 'unknown'. Why?
> > -> Suggestion: Raise bug and fix UI :)
> >
> > 9. Storage volume device name
> >
> > FGCP doesn't allow the user to specify the device name, nor can it
be
> > retrieved.
> > -> Suggestion: Make device attribute optional?
> >
> > 10. Storage snapshot creation does not return an id straight away
> >
> > The FGCP storage volume backup operation does not return an id
straight
> > away. On the FGCP Portal it does, with details of the progress of
the
> > snapshot creation process, but no API equivalent.
> > The only way to find out the id is to keep listing the backups of a
> > volume until the new entry appears.
> >
> > -> Suggestion: Make the snapshot id variable optional. Additionally,
we
> > could consider implementing
storage_snapshots(opts[:created(_since)])
> so
> > the user has a unique url to poll for it. Snapshots of a particular
> > volume cannot be created simultaneously so it should be accurate.
> >
> > 11. Address creation requires realm id
> >
> > In the FGCP address are created into a particular realm. How is it
for
> > other providers?
> > My driver currently just picks the first realm (VSYS) that the API
> > returns if :realm_id is not specified.
> >
> > Suggestion: Make it a feature so that we can extend the GUI to allow
the
> > user to select the realm from a list as is done for instance
creation.
> >
> > 12. Address creation does not return new IP straight away
> >
> > The FGCP address creation operation does not return an id/ip
straight
> > away. The only way to find out the new ip is to keep listing the
> > addresses until a new entry appears.
> >
> > -> Suggestion: Make the id optional?
> >
> > 13. Address association
> >
> > For performance reasons the addresses my driver returns for
addresses()
> > does not include the instance_id's they are assigned to.
> > I do include this when a single address is queried. The GUI lists
all
> > addresses as 'Free'. How can we implement a third state ('not
> > checked-drill down for details') for attributes like this? Use an
empty
> > string?
> >
> > 14. Address lifecycle mapping to DC
> >
> > Addresses in FGCP require more steps until they can be used compared
to
> > what DC expects:
> >
> > FGCP:
> > 1. Create address in realm (VSYS)
> > 2. Enable address
> > 3. Attach address to instance
> >
> > 4. Detach from instance
> > 5. Disable address
> > 6. Destroy
> >
> > DC:
> > 1. Create address
> > 2. Attach
> >
> > 3. Detach
> > 4. Destroy
> >
> > As each of the operations are asynchronous, I can't easily merge
three
> > into two (two into one) unless I introduce sleeps and polling.
> > Any advice on how to resolve this?
> > I currently combine the enable with the attach and the disable with
the
> > destroy. This means the first time you do an associate it fails with
an
> > error, but works if you try again after a while.
> >
> > -> Suggestion: Introduce enable & disable action/feature either on
all
> > providers, implemented as NOP where not relevant, or FGCP feature?
> >
> > 15. Firewalls mapped to instances and VSYS
> >
> > In the FGCP a firewall is automatically created as part of the
creation
> > of a VSYS. The FW needs to be started (it is actually implemented as
a
> > special VM in your VSYS) before you can register any rules. The
driver
> > therefore includes it in the instances list so you can start and
stop
> > it.
> > Can we make it a feature that some providers allow FW creation and
rule
> > creation in one operation, which FGCP won't support?
> >
> > In FGCP, when creating a VSYS, you have to specify a VSYS
descriptor.
> > The VSYS descriptor defines whether the VSYS is a 1, 2 or 3 tier
network
> > and can include server instance descriptions (i.e. you could have a
LAMP
> > VSYS descriptor that would allow you to create a 2 server - one in
DMZ
> > and one in SECURE zone - system created in one operation).
> > So I've mapped FW creation to VSYS creation using the FW description
as
> > descriptor. Should we introduce a separate feature/attribute
instead?
> >
> > The FGCP FW rules also don't map well in all cases to DC's. I'll
save
> > that for a separate e-mail. (Or maybe you can have a look yourself
on
> > the FGCP portal?)
> >
> > 16. Load balancers mapped to instances and VSYS
> >
> > The load balancer on FGCP needs to be started (it is actually
> > implemented as a special VM in your network segment) before you can
> > register any rules. The driver therefore includes it in the
instances
> > list so you can start and stop it.
> > Can we make it a feature that some providers allow load balancer
> > creation and rule creation in one operation, which FGCP won't
support?
> >
> > The FGCP load balancer rules also don't map well at all to DC's
> > listeners. I'll save that for a separate e-mail.
> >
> > That's it for now.
> > Cheers,
> > Dies Koper
> >
>
