Hi, here's the start of the static file generator. It just works on one web path, and one fileout at a time so far. It doesn't figure out the correct path to put the file, or check to see if there are any changes.
http://rene.f0o.com/~rene/stuff/pypi/pypi-static-generation.py # here is like looking at the http://cheeseshop.python.org/pypi/pygame url python pypi-static-generation.py -create_single /pypi/pygame /tmp/pygame.html It uses the webui.py code, so that there will not be any repeating of code. It does this in a similar manner to how the pypi.py pypi.cgi and pypi.fcgi codes works. That is by making its implementation of the RequestWrapper class. I thought I'd just keep posting my changes to the mailing list as I go... so there's some history of changes - and so people can have a look/review if they want. If that annoys people I'll stop sending to the list. Next up I'm going to put a few functions into store.py. Ones to check if a release has changed since a given date. Also one to see if any changes at all have happened since a given date. I'll also add some onChange type functions for releases. That will be where all of the code can go for stuff that happens on a change to releases etc. cheers, On 7/8/07, René Dudfield <[EMAIL PROTECTED]> wrote: > Hello, > > Cool, ok. Let's start with event based updating of the static files. > > > I need to make this tool in this way anyway though. But we can either > set it up to work with polling, or event based. We can start with > event based and switch to polling later if needed. > > Since none of the files exists at the moment, the tool will be > needed to generate them initially. Also if templates change, or the > database changes - then the static pages may need regenerating. > > > Polling is just one sql statement to see if something has changed. > You do this once, no matter how many things have changed. It's a > really quick, operation if nothing has changed. > > Polling ends up being faster if you are constantly having to do things > all the time anyway. It's what network drivers do these days because > they realise that there are a constant stream of events(interupts) > anyway - so might as well deal with them at a fixed interval. > > Logged in users will not see the static file anyway - since they are > logged in, they get to see the dynamically generated stuff. > > Imagine this case: > 2-3 users are updating their packages, at a similar time. The main > index then gets regenerated 3 times, rather than once. The more > people who are changing things the more this method works. If there > are 20 people changing things at the same time, then there is still > only one update of the main index page. However since the cheeseshop > only gets updated about 6 times daily, event based is probably better > for the moment. > > > Anyway... I'm just making the tool which can be used on demand, or at > regular timings. > > > Cheers, > > > > On 7/8/07, Jim Fulton <[EMAIL PROTECTED]> wrote: > > > > On Jul 7, 2007, at 12:24 AM, René Dudfield wrote: > > ... > > > Now I just need to finish off the static file generation code. It > > > needs a tool which can run every minute or so, which will look for any > > > changes. > > > > Why not write the files when the underlying packages change? > > > > I don't like polling for two reasons: > > > > - New pages are out of date for up to the polling interval. This is > > especially annoying for someone who uploads a package and wants to be > > able to access it immediately. > > > > - Polling all of the pages to see what's changed doesn't seem > > scalable to me. > > > > ... > > > > > I've also updated the http://wiki.python.org/moin/CheeseShopDev page > > > with some things I noticed when installing the cheeseshop again on my > > > laptop. Mainly dependencies, and missing config steps. > > > > Thanks! > > > > Jim > > > > -- > > Jim Fulton mailto:[EMAIL PROTECTED] Python > > Powered! > > CTO (540) 361-1714 > > http://www.python.org > > Zope Corporation http://www.zope.com http://www.zope.org > > > > > > > > > _______________________________________________ Catalog-SIG mailing list [email protected] http://mail.python.org/mailman/listinfo/catalog-sig
