Sorry, I missed this one earlier. We did discussion capabilities a bit, but we didn't really come to a conclusion. Besides, that people were more interested in talking and "fighting" about async support :-)
Another thing which I didn't mention in my first post and we discussed quite a bit is providers with multiple locations. We mostly agreed that new datacenter approach is better, but far from ideal and there is still a lot we can to to make it better. One of the options we discussed is using a special class instead of a string value for the "datacenter" argument. This class would contain all the logic and information which is required to talk to the provider (api endpoint urls, supported locations, etc.) There were also a lot of other ideas flying around. There are too many to sum all of them up in this post, but I will try to write a slightly longer blog post where I will dive deeper into the ideas we have discussed. On Wed, Jun 12, 2013 at 12:23 PM, John Carr <[email protected]> wrote: > Hi Tomaz, > > Thanks for sending this for those of us that couldn't make it! I've made a > few comments inline below. > > Did you discuss capabilities at all? > > Cheers > John > > On 12 Jun 2013, at 06:59, Tomaz Muraus <[email protected]> wrote: > > > First of all I want to say thanks to everyone for coming. We had a lot of > > good debates and received some first hand feedback from our users. > > Hopefully most of this debates will result into some actual code soon :) > > > > I've included a short recap and some action items bellow. Each section > > first contains a short problem description and a solution we agreed upon > > afterwards. > > > > *Pricing data distribution* > > > > Currently we bundle JSON file with provider pricing data with every > > release. There is no documented and easy way for user to update this file > > or for us to ship an updated version without releasing a new version of > > Libcloud. > > > > We should provide easy way to update this file which includes the > following: > > > > * Command line tool for updating the file - user can specify a custom URL > > to the file. By default it should point to the latest version from the > > version control ( > > > https://svn.apache.org/viewvc/libcloud/trunk/libcloud/data/pricing.json?view=co > > ) > > * Modify the code so it also tries to find the default pricing data file > in > > /home/user/.libcloud_data/pricing.json (or whatever) directory > > * Document this process and add it to the documentation page > > > > * Note: Most of code for this functionality already exists ( > > https://github.com/apache/libcloud/blob/trunk/libcloud/pricing.py). We > just > > need to add a code for downloading a file from a remote server and the > cli > > tool. > > It would be good if apps using this could control where the cached file is > kept. I'd want my scripts/apps to be able to update the file directly via > the API too, without needing to use the command line tool. > > > *Exception reporting for partial failures in the methods which result in > > multiple API calls / HTTP requests* > > * > > * > > Currently we have no standard interface for exceptions which get raised > > during a partial failure in a method which has side affects. Partial > > failure means that a function which performs multiple API calls failed > half > > way through and this potentially resulted in some (but not all) resources > > getting created. > > > > We should provide a special exception for cases like this. This exception > > should contain information about resources which got created. Users can > > then use this information to perform "rollback" / cleanup partially > created > > resources. > > What about having an addCleanup type stack like unittest has? > > Regardless of partial success it would be good if there was a mechanism > for having "it failed, but it is safe to retry" exceptions. > > > *Support for the async API in the core* > > * > > * > > Currently we only expose sync / blocking API. This mostly works OK for > > simple command line scripts, but it's less than ideal for long running > > process or applications which performs a lot of API calls which could be > > made more efficient and faster if we supported an async api. > > > > As part of this change we would eventually need to refactor every driver > > which would also provide other side benefits such as: > > > > * More maintainable code (smaller functions) > > * Easier way to handle partial failures / exceptions > > > > We didn't come to a consensus on everything, but we agreed on a couple of > > things: > > > > * Change should be backward compatible, we should default to sync API and > > existing behavior code / shouldn't change > > * New async API should be available separately (e.g. under > libcloud.async.* > > or whatever it ends up being) > > * We should refactor internal code to follow new interfaces defined in > the > > PEP 3156 (http://www.python.org/dev/peps/pep-3156/) > > * We should start with a small scale experiment and try this in a single > > method in a single compute driver > > Doesn't PEP 3156 depend on 'yield from' (i.e. python 3.3)? > > > *Documentation* > > > > We are weak on the documentation side. > > > > Going forward we should strictly enforce that every patch which adds new > > code / functionality contains documentation and appropriate docstrings. > > A related weakness is enforcing the interfaces we have defined, something > I hoped to address as part of: > > https://issues.apache.org/jira/browse/LIBCLOUD-289 > > The idea here is that the tests are more about testing the interface, and > less about testing the implementation. Then each implementation has a stub > test that tests each individual implementation conforms to the interface. > > As you can see in the diff, it identified cases where different > implementations had different return values and simple things like where > ports here strings instead of integers. > > > *Migrating to git* > > > > We want to make contributing as easy as possible and SVN doesn't help > with > > that and increases barrier to entry. > > > > Action item here is me opening an Apache Infrastructure ticket for > > switching to git. > > > > *Dropping support for Python 2.5* > > > > Supporting Python 2.5 adds code complexity which we would like to avoid. > > Main problem is that a bunch of CLI tools based on Libcloud usually also > > run on older versions of Linux (e.g. RHEL 5) which still ship with Python > > 2.5. > > > > We didn't managed to actually talk about this one, but we should explore > an > > option of removing support for Python 2.5 in the future release. > > > > If you have attended the meetup, feel free to share your notes and or / > > feedback. For other people who didn't attend - you are still more than > > welcome and encouraged to share your feedback with us here on the mailing > > list. > > > > - Tomaz > >
