Re: [Web-SIG] more comments on Paste Deploy
On Saturday 03 March 2007 15:54:41 Ian Bicking wrote: > Chad Whitacre wrote: > > I suggest that a system with multiple simple config files is much > > more scalable than a single complex config file syntax. Imagine > > if all of Unix were configured using a single syntax! > > There's other cases where having both options is nice. Because Paste > Deploy doesn't fold config files together, you can also reuse them from > different contexts. (A more common way to use multiple config files -- > what ConfigParser.load supports -- is to just overlap all the sections, > usually totally clobbering each other. I like this more explicit way of > bringing in configuration, which treats configuration like a composable > set of configurations instead of a system where all the configuration > files are pretty tightly bound to each other.) I find that multiple files gives you a nice way to override defaults. As long as the files are read in a way that's predictable and documentable, and ultimately appear as if read from a single file (and possible displayable via some diagnostics link in an application). -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] daemon tools
On Saturday 03 March 2007 11:08:24 Jim Fulton wrote: > > Anyway, I share this for your consideration. There are probably > better tools out there than zdaemon and supervisor2, but I'm not > aware of them. :) I'm curious what other people have found or use. ll.daemon (http://www.livinglogic.de/Python/daemon/index.html) seems to be a straightforward and very simple library for core daemon functionality. Daemontools isn't very well respected by the SysV style initscript crowd, and vice versa. That's an external non python dependency, and not commonly available. Certainly not available on Windows. I have written my own daemon base class (Pretty restrictive license [reciprocal], but I'm sure I could get it loosened). http://hg.rpath.com/raa-1.1?f=9ac380d082f4;file=raa/service/daemon.py I'm not married to it though, so would be happy to spin it out and remove the conary requirements, or just junk it. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] [Proposal] "website" and first-level conf (was: more comments on Paste Deploy)
On Saturday 03 March 2007 23:27:29 Chad Whitacre wrote: > 3) Common web app server > > Without discouraging the first two efforts, I'd like to champion the > third. Here would be my proposal: > > First, we define a "website" on the filesystem as a Unix-y userland > with, at minimum, the following: > > etc/.conf > lib/python > > Are you guys interested in this proposal? If so, I can write it up in > more detail. No, and here's why. Most apps are deployed as eggs. This is a relatively high ante to pay for complicated setups, but boilerplate setup.py code solves the 80% case well enough. Using eggs means that the apps could be installed in different locations, site-packages, user's own pythonpath, anywhere. The config file or files are going to be what determines what gets loaded, and where, much more than os.getcwd(). The configuration can be determined via searching well known locations /etc/foo.cfg ~/.foo.cfg ./foo.cfg, etc. or passed in on the command line.[1] References to apps will be to their eggs, which will be loaded from the Python path. Installing eggs to arbitrary file system locations, while it can be done, doesn't lend itself to super-packaging (rpm, dpg, installshield, etc). It also requires more setup by the end user/deployer than just running ez_install foo_app, or rpm -i foo_app.rpm. Also, user-land servers are not that interesting to me. They're great for development, but production use is where I see the pain. I'm interested in a common app server platform that focuses on running one or more applications from an egg (which could, and perhaps should include it's own configuration) mounted at different url locations. [1] This use of current working directory for configuration file loading could be used in the specialized aspen case for an exploded egg in an arbitrary file location. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] more comments on Paste Deploy
On Monday 05 March 2007 16:19:14 Ian Bicking wrote: > Joseph Tate wrote: > > I find that multiple files gives you a nice way to override defaults. As > > long as the files are read in a way that's predictable and documentable, > > and ultimately appear as if read from a single file (and possible > > displayable via some diagnostics link in an application). > > Allowing this sort of thing means that the application carries around a > complete config object of some sort, which I rather dislike -- it allows > for smart applications, but it makes it much harder to understand the > configuration and any possible side effects. If we resolve the > configuration down to something more limited (as the Paste Deploy entry > points do) you can't really reconstruct the config from there. > *Something* could still reconstruct the config (an alternate config > loader, via logs, via debug settings, etc), just not the application > itself. > > This is somewhat problematic for applications that have particularly > complex config requirements, or want to support self-configuration. The > best solution that I can think of with Paste Deploy in that case is to > just use the Paste Deploy configuration to point to the "real" > configuration. I agree. That's why my app has a /config link that spits out the "effective" configuration. The overridden config is a hard requirement, I'd love to hear alternative solutions. /etc/php.d, /etc/httpd/conf.d and that ilk come to mind as examples of this kind of thing. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] more comments on Paste Deploy
On Monday 05 March 2007 16:38:51 Phillip J. Eby wrote: > At 10:02 AM 3/5/2007 -0500, Jim Fulton wrote: > >Entry points add *a* mechanism to make those objects a bit more > >discoverable. Arguably, specifying an application via: > >eggname#entrypointname doesn't provide much advantage over simply > >specifying the dotted path to an object in a module. > > Actually, it provides one very important strategic advantage that I don't > think has been mentioned in this conversation. A configuration format that > can specify project/version information can be used as a single-file > deployment spec for an easy_install wrapper or buildout-like tool. > > The advantage of this for virtual hosting providers in particular is > significant -- if they support the tool, they can support this one-file > deployment scheme. > > Personally, I don't care for the Paste Deploy syntax -- frankly it's almost > barbaric. :) But the concept of being able to specify stacks, routes, and > configuration in a plain text format that includes package information for > automated deployment is nonetheless an important one. > > A couple years back, I started writing a library to parse a more > sophisticated, Python-like syntax to do the same sorts of things, but only > got as far as the parser. > > One discussion was here: > > http://mail.python.org/pipermail/web-sig/2005-August/001714.html > > The basic idea behind the syntax was that assignments are like keyword > arguments, and non-assignment statements are positional arguments. > > I'm not altogether happy with that syntax either, however, as it has a > little too much "more than one way to do it", which is one reason I never > finished the implementation. There is a library that parses it (and does > other general-purpose Python-like DSL parsing) at: > > ViewSVN: http://svn.eby-sarna.com/SCALE/ > Checkout: svn://svn.eby-sarna.com/svnroot/SCALE/ > Docs: > http://peak.telecommunity.com/DevCenter/scale.dsl#parsing-declarations > > Anyway, all that aside, I think it would be fantastic if we could come up > with some "universal file format" for single-file configuration and > deployment of applications (including auto-install of all needed eggs), > that could get stdlib support and ultimately hosting company support. This > would actually give us a leg up on even PHP for ease-of-deployment. Doesn't setuptools already give this? easy_install foo.app.egg will install all of the needed eggs if the dependencies are properly listed. > So, while a single content format would be nice, we don't even need that in > order to get a raw deployment system standard. Perhaps I should build this > hypothetical pydeploy tool into setuptools 0.7? I don't see there being a lot of demand for this. The use case I'm considering is the end user developer or administrator deploying one or more delivered pyhon web applications to a production environment (self hosted, colo-hosted, or leased server). I think that except for where you have multiple servers behind a load balancer or something, this is a one time operation (barring failure cases, etc). Administrators already script this kind of thing using shell. Also, in any "enterprise" environment that I'm familiar with, the automatically download and install software mechanism wouldn't fly. Administrators want to know everything that goes on a system, and want the software managed through their patch/package management system. Philosophical discussions on whether that's good or not seem to be irrelevant. Those using $4.95 hosting plans are only setting up one server, and will need something custom to their installation anyway, so "pydeploy" won't help them either. They'll be trying to install trac, some blogging software and then an arbitrary image gallery, et. al., but won't have the same selections as another $4.95 hosting customer. This is the key problem we're trying to solve. I consider the packaging and delivery problem solved[1], or at least out of the scope of this problem. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 [1] Good enough for most things but better support for stuff outside the egg is needed: config files (so that the user can tweak them), locale data (or maybe a pkg_resources wrapper for gettext that loads that data from the egg). ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] more comments on Paste Deploy
On Wednesday 07 March 2007 05:34:15 Jim Fulton wrote: > I'd love to get some input who know a lot about what makes deploying > PHP apps so easy. It's not the packaging format. Most php apps come down as a tarball. Extract it to your apache root, and you can connect to the app and do configuration, without even restarting apache (thanks to mod_php). I think the key thing is that configuring a "well written" php app is done through the web interface. No mucking with config files, no apache configuration required, etc. Just have to create a database and a user with permissions to it. If you'd like a specific example, I suggest trying to install gallery (http://gallery.menalto.com). There are sacrifices to make for this approach though: the app has to be able to write at least to its own config file, and to .htaccess. This means that security has to be super tight. Frequently the instructions are to chmod 777 the app's top level directory, configure, and then unchmod. Because so many things can be modified via .htaccess, including directory specific php settings, you rarely need further configuration. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Web Site Process Bus
On Monday 25 June 2007 20:32:38 Chad Whitacre wrote: > All, > > >> Now I'm really confused. What is the idle state *for*? > > > > One concrete use case could be a test suite ... > > Is this also where Joe's comment in Dallas fits, namely, the case > where you want to hold open a socket while restarting a child > process so as not to drop connections? It's not so much a desire to not drop connections as to keep the socket file descriptor during a restart to prevent an asynchronous kernel call (socket.close()) from preventing your new process from reopening that socket. The frequent work around is to insert an arbitrary sleeps or loops on the socket.bind call, but I store the file descriptor as an environment variable and reconnect to the existing socket after the exec call. This is a problem on Linux at least, and probably all POSIX systems. Having the kernel queue up connections is an added bonus. I do think this is a good example of what the IDLE state could be used for. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
Re: [Web-SIG] Web Site Process Bus
On Monday 25 June 2007 22:48:47 Graham Dumpleton wrote: > Thus I believe a clear statement should be made that UNIX signals are > off limits to WSGI applications or components. I have no problem with > the underlying server upon which the WSGI adapter sits using them for > its own purposes, however, knowledge of signals should stay in that > layer. Now it may be the case that a higher level application signal > may be generated as a result of a SIGTERM in a specific server > implementation, for example shutdown, but the name or type of signal > does have to be distinct, ie., "shutdown" and have no linkage to the > UNIX signal. I think this is fine. Signal handlers only get run in the main thread anyway, so ignoring them for sub-threads (like handlers, dispatchers, etc) is a natural division. A web bus obviates the need to signal other processes in the process group. I'm not clear on how the web bus devolves in the multi-process state though. My particular use case for keeping SystemExit around is I have an app that allows the user to upload a new SSL certificate. Without a restart, or perhaps, given a WSPBus, just a drop to IDLE state, the new SSL certificate would not be applied to new incoming connections. The thread that puts the new SSL Certificate in place needs to be able to tell the entire server to reload. A second use case has already been mentioned: reloading running code. The same app as above runs the system software update mechanism. When that update completes (which could contain a new core c library, new python, new modules, new application code, or even new configuration data) the server needs to reload. I think that SystemExit should not be removed from the WSGI application layer, unless it's replaced with another mechanism that allows the working thread/process to signal that the entire server should be restarted/reloaded. -- Joseph Tate Software Engineer rPath Inc. http://www.rpath.com/rbuilder/ (919) 851-3984 x2106 ___ Web-SIG mailing list Web-SIG@python.org Web SIG: http://www.python.org/sigs/web-sig Unsubscribe: http://mail.python.org/mailman/options/web-sig/archive%40mail-archive.com
