[
https://issues.apache.org/jira/browse/LIBCLOUD-275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543603#comment-13543603
]
Tomaz Muraus commented on LIBCLOUD-275:
---------------------------------------
I just want to add some context to this debate and let everyone know the
reasoning around the current base APIs.
Currently most of the libcloud methods directly map to a single HTTP request.
The main reason for this is that is keeps things simple because each method can
be considered "atomic" - it fully succeeds or it doesn't (e.g. a server is
created (success) or it isn't (error is thrown)).
If a method fails / throws an exception you can easily retry it, but if a
method performs multiple actions (multiple HTTP requests), your stuff could end
in an inconsistent state.
For example, lets have a look at EC2 compute driver create_node method which
currently performs two HTTP requests if you create a single server:
1. Create a server
2. Create a special "name" tag which is treated as a server display name
In this case first request could succeed, but second one could fail (we have
already been "burnt" by this because it happens relatively often). If this
happens, we throw an exception for the tag creation part. Exception doesn't
contain any reference to the resource which have been successfully created (a
server / node in this case), which makes it harder for you to clean stuff up.
Overall I am for utility methods because they make things simpler, but we need
to be careful and come up with a convention or something on how to handle /
propagate partial failures in those methods and also document this well so user
knows what to expect if one of this methods is used.
It could be something as simple as putting more attributes on the exception
(e.g. a list of "resources" which have been created) and than it's up to the
user to perform cleanup manually.
> Utility APIs for containers
> ---------------------------
>
> Key: LIBCLOUD-275
> URL: https://issues.apache.org/jira/browse/LIBCLOUD-275
> Project: Libcloud
> Issue Type: New Feature
> Components: Storage
> Reporter: Mahendra M
> Assignee: Mahendra M
> Priority: Minor
>
> While working with libcloud, I thought it will be good to have some utility
> APIs to containers, which will help in easier use. Some APIs will have a base
> class implementation, which can be easily over-written in each driver based
> on the storage provider.
> Some APIs that I have in mind are:
> 1. container.copy_object(obj, target_container)
> Copies an object from one container to another. Base class implementation can
> be
>
> target_container.upload_object_via_stream(container.download_object_as_stream(obj),
> obj.name)
> In each driver, this can be over-written to use specific methods to do this
> (AWS, Azure etc. provide APIs for doing this).
> 2. container.move_object(obj, target_container)
> Base class implementation can be:
> container.copy_object(obj, target_container)
> obj.delete()
> Again, driver specific extensions can be done.
> 3. container.sync_to_local(local_path, only_missing=True, only_newer=False)
> 4. container.sync_from_local(local_path, followlinks=False, prefix='',
> only_missing=True, only_newer=False)
> The above two functions are good for syncing local folder repos with cloud.
> The functions are ridiculously easy to implement and will provide good
> mechanism for sync. (For eg: syncing a folder which contains 500 HLS chunks
> of a video)
> The parameters used are:
> * only_missing - sync only files that missing in the target (container or
> folder)
> * only_newer - sync only files that have last-modified (or m_time) more than
> the target
> * prefix - A prefix to be prefixed to each object name while syncing from a
> local folder.
> I am just opening a ticket so that we can discuss this and finalize it. Once
> everyone agree, I can submit/check-in a patch
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
