On Fri, Jul 8, 2011 at 3:24 PM, Brian Bouterse <bmbou...@gmail.com> wrote:
> I've been thinking about this for a day or two now, and here are some > thoughts: > > One option to possibly compile python to C++ is shed > skin<http://code.google.com/p/shedskin/>, > to translate python code into a C binary. Note that the compiled C code is > statically compiled which produces a C instruction stream that is static > (independant of the actual control flow). I haven't run shed skin myself, > and there are language restrictions shed skin requires that would need to be > looked into since I'm not familiar with django's comatibility with shed > skin. > Shedskin also looks interesting, but worries me slightly on it's lack of known compatibility with Django. > > Note that the statically built C code might be less efficient than a Just > In Time compiled counterpart. I have used pypy <http://pypy.org/> before > which makes python code run fast <http://speed.pypy.org/> and run my own > speed tests versus python code run through CPython, and pypy significantly > (over double typically) outperforms the same code run through CPython. What > you would get with pypy would not be a single, pre-compiled binary, but the > speed ups of running django (and potentially nginx and gunicorn, etc) > through a pypy interpreter could potentially outperform a statically built > one. After all the PHP to C translation speedup that facebook claims is > just over a 50% increase (anecdotally told in the video), pypy already does > that for me versus the CPython. I have also heard of folks using pypy in > production serving django sites, so I think Django is ready for this. > Wow, I didn't know about pypy, those benchmarks are very impressive, and should certainly be considered for a project like this. > > I think also value the bundling of the web server, static content, and > application together is the value so that you don't have to manage the > stack. Requiring it to be a *single* binary is nice, but I don't think buys > you much really. After all you could always just tar up the entire project > and deliver it that way. > Another possible issue about compiling everything as a single binary, is that we could get very strange compatibility problems between components. An easier way would be to compile a single compressed binary (containing a tar with the chroot/stack/webapp), which then self extracted into memory. I'm torn between both to be honest. A true single binary is cleaner, but a self extracting binary is easier. If we opt'd for having a true single binary, there would defo need to be some other experienced C coders on the project, as my knowledge of C is weak and progress would be painfully slow lol. > > What still needs to be figured out is how to get pypy reading the nginx and > gunicorn C code natively and then switching over and using JIT compilation > on the python code from the gunicorn->django entry point. > One of the principles is to eventually allow this to work with other languages (such as PHP or Ruby), so we can't lock it down too heavily to Python. > > I would love to hear what others think about this type of thinking. > > Best, > Brian > > On Fri, Jul 8, 2011 at 5:55 AM, Cal Leeming [Simplicity Media Ltd] < > cal.leem...@simplicitymedialtd.co.uk> wrote: > >> Also, one of the things you mentioned (about security updates of the >> chroot) is something i have been pondering today. Although a single compile >> will mean recompile everytime a security announce is released, this is >> unavoidable. >> >> This is my idea for handling security announcements: >> >> The single binary will have support for monitoring various sources for >> indications as to whether or not there is a security announcement for one or >> more of its internal componants. Within the configuration, the user can set >> an email address which these alerts are sent to. To avoid any single person >> being the weak link, we'll make it monitor not only the official project >> announce, but also remote security repos, to give it hints as to whether or >> not there is a security release for any specific component. >> >> The above idea will need some tweaking ofc, or maybe someone else could >> think of a better way to do this..? >> >> This also makes me think about build control. I guess each project would >> end up having a build file for future rebuilds, would it be wise to >> integrate this into the binary, as a backup incase the original is lost?? >> >> So many thoughts.. >> >> Cal >> On 8 Jul 2011 10:47, "Cal Leeming [Simplicity Media Ltd]" < >> cal.leem...@simplicitymedialtd.co.uk> wrote: >> > Hi Bjarni, >> > >> > Thanks for your input. As pointed out earlier in the thread, the end >> goal is >> > to provide a single binary that is entirely host independant, contains >> the >> > chroot/stack needed by the webapp etc. >> > >> > You really need to watch the facebook video link i included earlier in >> the >> > thread to understand what we're trying to achieve. >> > >> > Im drawing up the design principles today, which should give you a >> better >> > idea.. but in a nutshell, imagine a single binary which would run >> entirely >> > from memory, be completely host independant, contain everything the >> webapp >> > needed (stack/chroot/deps etc), and have a nice ncurses interface to >> > configure this all from (with features like automatic memory usage >> > calculation / configuration - something which is overlooked oh so many >> > times) >> > On 8 Jul 2011 08:53, "Bjarni Rúnar Einarsson" <b...@pagekite.net> wrote: >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Django users" group. >> To post to this group, send email to django-users@googlegroups.com. >> To unsubscribe from this group, send email to >> django-users+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/django-users?hl=en. >> > > > > -- > Brian Bouterse > ITng Services > -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
