Re: [GSoC 2013] contribute to django-deployer for deploying django to PaaS easily

[Nate Aune]

On Thursday, May 2, 2013 4:11:35 AM UTC-4, Andrew Godwin wrote:
>
> I feel like the deployment problem is one that cannot be solved in a mere 
> 12 weeks and I'm not sure django-deployer is the right approach to 
> encourage for this - it sits at the wrong level of abstraction and looks 
> pretty fragile, and I'd be hesitant to put any sort of official emphasis on 
> it without a lot of my qualms being answered. 
>

Could you elaborate on what your qualms are? I won't claim that it's a work 
of art and I know there is lots of room for improvement, but please 
remember that django-deployer is the result of two days of work, and it 
already deploys Django projects to Dotcloud and Google App Engine with 
almost minimal effort on the part of the user. Imagine what we can do in 12 
weeks!
 

> In particular, it appears to be sitting at the lowest-common-denominator 
> level of the various feature sets and looks pretty immature.
>

The initial goal of django-deployer was to reduce the barrier to entry for 
someone new to Django who wants to quickly and easily get their Django 
project running up on one of the PaaS providers.
So we started out with a minimum feature set which seems to cover the 
non-production deployment needs for 80-90% of Django projects. 

We purposely omitted support for services such as Memcached, Celery, email, 
cron jobs, etc. because we were trying to limit the scope at the sprint and 
get a basic version working first. Now that the basic version is working, 
we can look into adding support for these other services that you'd most 
likely find in a production environment. 

I'm not sure I could accept this without a much clearer idea of what's 
> going to happen and a very clear focus on what section of our user base it 
> will help. Deployment isn't something there can be a single solution (or 
> even pattern) for, and I think encouraging that could be a bad idea.
>

Agreed. We're not attempting to make a one-size-fits-all solution, and 
django-deployer will never claim to serve that purpose. But for 
bootstrapping your Django project so that you can minimize the amount of 
time it takes to try deploying your project to various PaaS providers, and 
for demonstrating best practices when readying your project to be deployed, 
I think it has a place in the Django ecosystem. 

I've looked at all the getting started docs from the various PaaS 
providers, and many of them leave a lot to be desired. Many developers that 
I've spoken with who've tried PaaS have thrown up their hands in 
frustration when things don't work as advertised. This is largely the 
impetus for starting the django-deployer project, to try to streamline this 
process for those new to PaaS and Django deployment.

Nate


> On Wed, May 1, 2013 at 8:20 PM, Nate Aune <nate...@gmail.com 
> > wrote:
>
>> Hi Russell and Florian,
>>
>> A bit late to the party, but hopefully there's still time for this GSoC 
>> proposal to be considered. I'm the maintainer of 
>> django-deployer<http://natea.github.io/django-deployer/>. 
>> It was initiated at the DjangoCon 2012 sprint and recently revisited at the 
>> PyCon 2013 sprint. The overarching goal of django-deployer is to make it 
>> easier to get Django deployed. So far the focus has been on deploying to 
>> PaaS providers, because they require little to no sysadmin skills, and have 
>> the added benefit of being free for small projects.
>>
>> django-deployer is the sister project PaaS Cheatsheet, a larger 
>> documentation project that I've started recently. PaaS 
>> Cheatsheet<http://www.paascheatsheet.com>is a guide for how to get a 
>> Django/Python deployed on various PaaS 
>> providers. 
>>
>> The way django-deployer works is by asking a series of questions about 
>> your Django project, and then generates a generic deploy.yml file that 
>> captures all of your project's requirements. When you choose a particular 
>> PaaS provider, django-deployer then translates the requirements defined in 
>> the deploy.yml file and generates configuration files for that specific 
>> PaaS provider.
>>
>> I met Colin (a student in Taiwan) at the PyCon sprint, and he was 
>> responsible for single-handedly adding Google App Engine support to 
>> django-deployer. He brought considerable expertise to our sprint team, and 
>> shipped working code within a matter of hours. Since returning from PyCon, 
>> we've been working remotely together, and he has continued to be a 
>> dedicated contributor to the project. I have utmost confidence in his 
>> abilities to add even more cool features and improve code quality of the 
>> django-deployer tool if this project is accepted into GSoC. 
>>
>> English is a second language for Co

Re: [GSoC 2013] contribute to django-deployer for deploying django to PaaS easily

[Nate Aune]

On Thursday, May 2, 2013 9:46:52 PM UTC-4, Jacob Kaplan-Moss wrote:

> While I share some of Andrew's concerns, I do think this is still a fairly 
> good topic for GSoC. 
>

Thanks for your vote of confidence! :) We'd love to get your input on how 
to address Andrew's concerns. We've already started looking into replacing 
the manage.sh script with Django management commands, so you can deploy 
using the familiar manage.py (which is something that I think you suggested 
awhile ago). 

I made a couple screencasts where you can see the tool in action, in it's 
current state of infancy:
http://appsembler.com/blog/deploy-django-apps-to-google-app-engine-with-django-deployer-in-5-minutes/
http://appsembler.com/blog/deploy-django-apps-to-dotcloud-with-django-deployer-in-5-minutes/
 

> However, I think the only way it could happen would be if Nate wants 
> to mentor the project; Nate, can you mentor? 


Yes! As I mentioned in my last post, I'm happy to serve as a GSoC mentor. I 
will do my best to provide guidance to Colin as we continue to shape 
django-deployer to be a useful tool for the Django community.
 

> If so, you should hook up  with Andrew and make sure you apply to be a 
> mentor so we can have you 
> added before the review process. 
>

I added my name to the wiki, and also signed up on the GSoC site 
here: http://www.google-melange.com/gsoc/connection/google/gsoc2013/nateaune/1

Nate

On Thu, May 2, 2013 at 4:11 AM, Andrew Godwin 
<and...@aeracode.org> 
> wrote: 
> > I feel like the deployment problem is one that cannot be solved in a 
> mere 12 
> > weeks and I'm not sure django-deployer is the right approach to 
> encourage 
> > for this - it sits at the wrong level of abstraction and looks pretty 
> > fragile, and I'd be hesitant to put any sort of official emphasis on it 
> > without a lot of my qualms being answered. In particular, it appears to 
> be 
> > sitting at the lowest-common-denominator level of the various feature 
> sets 
> > and looks pretty immature. 
> > 
> > I'm not sure I could accept this without a much clearer idea of what's 
> going 
> > to happen and a very clear focus on what section of our user base it 
> will 
> > help. Deployment isn't something there can be a single solution (or even 
> > pattern) for, and I think encouraging that could be a bad idea. 
> > 
> > Andrew 
> > 
> > 
> > On Wed, May 1, 2013 at 8:20 PM, Nate Aune <nate...@gmail.com> 
> wrote: 
> >> 
> >> Hi Russell and Florian, 
> >> 
> >> A bit late to the party, but hopefully there's still time for this GSoC 
> >> proposal to be considered. I'm the maintainer of django-deployer. It 
> was 
> >> initiated at the DjangoCon 2012 sprint and recently revisited at the 
> PyCon 
> >> 2013 sprint. The overarching goal of django-deployer is to make it 
> easier to 
> >> get Django deployed. So far the focus has been on deploying to PaaS 
> >> providers, because they require little to no sysadmin skills, and have 
> the 
> >> added benefit of being free for small projects. 
> >> 
> >> django-deployer is the sister project PaaS Cheatsheet, a larger 
> >> documentation project that I've started recently. PaaS Cheatsheet is a 
> guide 
> >> for how to get a Django/Python deployed on various PaaS providers. 
> >> 
> >> The way django-deployer works is by asking a series of questions about 
> >> your Django project, and then generates a generic deploy.yml file that 
> >> captures all of your project's requirements. When you choose a 
> particular 
> >> PaaS provider, django-deployer then translates the requirements defined 
> in 
> >> the deploy.yml file and generates configuration files for that specific 
> PaaS 
> >> provider. 
> >> 
> >> I met Colin (a student in Taiwan) at the PyCon sprint, and he was 
> >> responsible for single-handedly adding Google App Engine support to 
> >> django-deployer. He brought considerable expertise to our sprint team, 
> and 
> >> shipped working code within a matter of hours. Since returning from 
> PyCon, 
> >> we've been working remotely together, and he has continued to be a 
> dedicated 
> >> contributor to the project. I have utmost confidence in his abilities 
> to add 
> >> even more cool features and improve code quality of the django-deployer 
> tool 
> >> if this project is accepted into GSoC. 
> >> 
> >> English is a second language for Colin, so he may have had some 
> >> difficulties in defending his proposal, so please allow me to step in 
> and 
> >> clarify

Re: [GSoC 2013] contribute to django-deployer for deploying django to PaaS easily

[Nate Aune]

Hi Russell and Florian,

A bit late to the party, but hopefully there's still time for this GSoC 
proposal to be considered. I'm the maintainer of 
django-deployer. 
It was initiated at the DjangoCon 2012 sprint and recently revisited at the 
PyCon 2013 sprint. The overarching goal of django-deployer is to make it 
easier to get Django deployed. So far the focus has been on deploying to 
PaaS providers, because they require little to no sysadmin skills, and have 
the added benefit of being free for small projects.

django-deployer is the sister project PaaS Cheatsheet, a larger 
documentation project that I've started recently. PaaS 
Cheatsheetis a guide for how to get a 
Django/Python deployed on various PaaS 
providers. 

The way django-deployer works is by asking a series of questions about your 
Django project, and then generates a generic deploy.yml file that captures 
all of your project's requirements. When you choose a particular PaaS 
provider, django-deployer then translates the requirements defined in the 
deploy.yml file and generates configuration files for that specific PaaS 
provider.

I met Colin (a student in Taiwan) at the PyCon sprint, and he was 
responsible for single-handedly adding Google App Engine support to 
django-deployer. He brought considerable expertise to our sprint team, and 
shipped working code within a matter of hours. Since returning from PyCon, 
we've been working remotely together, and he has continued to be a 
dedicated contributor to the project. I have utmost confidence in his 
abilities to add even more cool features and improve code quality of the 
django-deployer tool if this project is accepted into GSoC. 

English is a second language for Colin, so he may have had some 
difficulties in defending his proposal, so please allow me to step in and 
clarify a few things.
 

> Firstly, I'm not familiar with Django-deploy; but from a quick inspection 
> of the project page it doesn't appear that anyone in the Django core team 
> is on the committee list. In order for you to have this proposal accepted 
> as part of the GSoC, you'll need to secure agreement from the project 
> maintainers of Django-deploy that they want the help your offering, that 
> your project proposal is sound, and that they're willing to commit to 
> applying any patches you produce. 


As stated above, I'm willing to work closely with Colin to meet the 
objectives described in the proposal. 

Secondly, you haven't provided any timelines for your proposal. My 
> immediate reaction is that you're proposing a lot of work for a student to 
> complete in 12 weeks. For example, writing a fully tested and documented 
> database backend strikes me as a 12 week project in itself - yet this is 
> apparently just one line item in one part of your project proposal.
>

I think you may have misread that part of proposal. He's not proposing to 
write a database backend.  There is already one provided by the Google App 
Engine 
SDK,
 
so we're simply incorporating that into the configuration scripts (i.e. 
adding a settings_appengine.py that substitutes the DATABASE setting 
value). 

>
> Lastly, your project proposal is very light on details. You're proposing a 
> lot of ideas, but you haven't really specified anything beyond "I'm going 
> to do it". Are there APIs that need to be specified? If, in the case of 
> something like a database backend, the APi is already specified, are there 
> going to be any complications in satisfying the API? We need a lot more 
> detail before we will be able to judge if you understand the project your 
> proposing, or if you are just proposing a bunch of ideas but haven't given 
> those ideas any more thought than "this sounds interesting"
>

I think the draft proposal was intended to get initial feedback and was 
intentially light on details. Colin and I can work on specifying more 
details on the actual deliverables within the next couple of days. We've 
already posted some ideas for improving it to the Github issue tracker: 
https://github.com/natea/django-deployer/issues?state=open

>You list "Refactor for Expandability" as last on your schedule. I think it 
should be at the start, eg compare different solutions like GAE, heroku, 
Gondor, find a >common subset and then write the backends accordingly. I 
fear that if you do this the other way around you will have to throw away 
most of the code for >heroku/GAE and rewrite it.

That's essentially what we've started to do on the PaaS cheatsheet site. 
One interesting side benefit of looking at all the PaaS providers, is that 
we've able to identify the commonalities among them.

>* Regarding storage stuff, you say "I chose Google Cloud Storage according 
to the networking speed and authorization flow will be easier than using 
other >storage service such as S3.". Personally I