I've been working a little on a proposal for some new cookbook sections, one per chapter, that provide a roadmap to the web2py source that's relevant to the chapter.
My motivation, mentioned below in the Introduction section, is that the full-stack nature of web2py is one of its important features, and can be used to advantage by developers. But the source is intimidating for newcomers, and even for experienced web2py developer's, it's not alway obvious where one should be looking. US cookbooks often have a chapter or two that talk about ingredients and techniques rather than recipes per se; the standard cookbook Joy of Cooking calls this chapter "Know Your Ingredients", and I've borrowed the name. My tentative decision is to distribute this material across the existing chapters, as the last section in each chapter. However, and argument could be made for making it a separate chapter. I'm ambivalent on the question. If you guys are agreeable, I'll proceed with the idea. Below are an introduction and the Use the Source section for Chapter 1. KNOW YOUR INGREDIENTS Introduction One of the less visible features of web2py is that it’s a fully integrated framework with a compact and readable source that’s part of your installation. When you have a question that isn’t answered in the web2py book, and isn’t directly addressed by one of these recipes, the answer can generally be found in the source. Each chapter in this book concludes with a Know Your Ingredients section, that serves as a guide to the portions of the source that make up the subject matter of the chapter. These sections are not detailed descriptions of the source, but are rather high-level roadmaps to assist your own navigation. Installation and Deployment Every incoming request to web2py is sent from the web server to gluon.main.wsgibase, which, as the name suggests, supports the WSGI application API. The means by which the request gets from the server to wsgibase is a function of the kind of deployment being used. Different paths require different startup or handler files by which web2py is run. We can divide the deployment methods into three categories: CGI deployment requires web2py to be run as a new process for each request. The CGI handler is cgihandler, which uses the wsgiref module’s CGIHandler to pass the request to wsgibase. This is a pattern that you’ll see repeated in many of the handlers: an API such as CGI is translated to a WSGI call and sent to wsgibase. The Google App Engine is CGI-like, in that web2py runs once for each request. GAE is configured via app.yaml to run gaehandler, which in turn uses standard Google libraries to configure the environment before invoking CGIHandler.py, or else running wsgiref directly if logging is enabled. Notice the configuration parameters near the beginning of gaehandler, and the environment variable SERVER_SOFTWARE a little farther on. web2py includes Rocket, a built-in pure-Python WSGI-compatible server that can directly handle web server duties, or act as the back-end server for a proxy such as Apache’s mod_proxy. When using Rocket, web2py is started with the web2py.py script, which simply calls gluon.widget.start to handle command-line options and optionally present the operator with a minimal Tk GUI. Once running, Rocket receives HTTP requests and passes them to Rocket. See gluon.main.HttpServer for the invocation of Rocket. Finally, web2py can run continuously and field requests from an external web server though the WSGI API (wsgihandler), the FastCGI API (fcgihandler) or mod_python (modpythonhandler). The WSGI handler is the simplest, because web2py is a native WSGI application. modpythonhandler wraps a mod_python interface around the call to wsgibase with help from mod_python.apache, while fcgihandler uses gluon.contrib.gateways.fcgi, a contributed module, to convert from FastCGI to WSGI, eventually calling wsgibase. But wait, there’s one more. A recent addition to web2py, the anyserver script, inspired by Bottle’s similar functionality, supports a long list of web servers—just about any server that supports the WSGI API. The list is too long to repeat here, but if you have a favorite server you’d prefer to use, be sure to browse anyserver to see if it’s on the list. Installation & Startup Scripts A few installation scripts can be found in the the scripts directory; look for scripts named setup-*, such as setup-web2py-ubuntu.sh. Likewise, a few startup scripts designed for integration into a hosts boot-time initialization method are found in the scripts directory: web2py.archlinux.sh, web2py.fedora.sh, and web2py.ubuntu.sh. Dispatching a Request The web2py request dispatcher, gluon.main.wsgibase, is where each incoming HTTP request ends up once it makes its way through whichever handler is being used. Here’s we’ll briefly describe the top-level flow of a request. main.wsgibase Initialize request-scope globals (request, response, etc) Rewrite URL (gluon.rewrite) Stream static files via streamer.stream_file_or_304_or_206. Notice that the ultimate handler of a request ends up raising an HTTP exception, which is handled farther down in main.wsgibase. Perform per-request housekeeping: parsing the query string, handling cookies, initializing the session and the response headers. main.serve_controller: dispatch the request to the application compileapp.build_environment: build the environment in which the models, controller and view will run. (This is where the request-scope globals such as URL, T, request, response are supplied.) Set the default view, based on the request’s controller, function, extension. Run the application’s models: compileapp.run_models_in Run the selected controller: compileapp.run_controller_in If the controller returns a dict, run the selected controller: compileapp.run_view_in. Notice that both the controller and view see the environment as modified by the models, but the controller does not affect the view’s environment. Raise an HTTP exception (typically 200 OK) to return control to main.wsgibase. except HTTP: send response to the server Static files are served directly. Commit database. You’ll notice later that if the request ended in an error, the database is rolled back. Save the session on disk. Create cookie headers finally: run cron if softcron, and respond via http.HTTP.to Finish creating the response headers. Send status and headers back via WSGI responder callback. Return body of response to WSGI caller.
