On 12 Sep, 12:44, Bruno Desthuilliers <bruno. [EMAIL PROTECTED]> wrote: > Astley Le Jasper a écrit : > > > I'm still learning python and would like to know what's a good way of > > organizing code. > > > I am writing some scripts to scrape a number of different website that > > hold similar information and then collating it all together. Obviously > > each site needs to be handled differently, but once the information is > > collected then more generic functions can be used. > > > Is it best to have it all in one script or split it into per site > > scripts that can then be called by a manager script? > > If everything is > > in one script would you have per site functions to extract the data or > > generic function that contain vary slightly depending on the site, > > As far as I'm concerned, I'd choose the first solution. Decoupling > what's varying (here, site-specific stuff) from "invariants" is so far > the best way I know to keep complexity manageable. > > > for > > example > > > import firstSiteScript > > import secondSiteScript > > > firstsitedata = firstSiteScript.getData('search_str) > > secondsitedata = secondSiteScript.getData('search_str) > > etc etc > > Even better : > > - put generic functions in a 'generic' module > - put all site-specific stuff each in it's own module in a specific > 'site_scripts' directory > - in your 'main' script, scan the site_scripts directory to loop over > site-specific modules, import them and run them (look for the __import__ > function). > > This is kind of a Q&D lightweight plugin system, that avoids having to > hard-code imports and calls in the main script, so you just have to > add/remove site-specific script to/from the site_scripts directory . > > Also, imported modules are not recompiled on each import - only when > they change - while the 'main' script get recompiled on each invocation. > > (snip) > > > OR > > > def getdata(search_str, website): > > if website == 'firstsite': > > .... > > elif website =='secondsite': > > This one is IMHO the very worst thing to do. > > My 2 cents...
Excellent, thanks for that. -- http://mail.python.org/mailman/listinfo/python-list