"pileux systeme" <[EMAIL PROTECTED]> wrote > I am trying to retrieve data from several webpages. > My problem is the following: after a random number of requests, > the page I'm trying to open is unavailable (and I get an IOError). > Note that the page may become available if I try again
This is similar to the problem faced by marketeers doing outbound phone calls. The customer may not be home so you have to try again later (maybe several times). The normal solution to this is to buoild a list of failed addresses and then after processing the original list go back and process the exceptions list. Repeat for N iterations or until exceptions list is empty. This approach saves multiple attempts in clise proximitry to a faulty address when you could be processing productive addresses so is time efficient. The only snag is if you have to process the addresses in a given sequence. In that case you may need to create a list of groups and if any address within a group fails put the entire group in the exceptions list. Coincidentally, given my recent message on Mainframe technologies, this is the same technique used in high volume data processing (on mainframes) where millions of orders or bills etc need to be processed quickly. Rather than delay the multitude to fix errors on a case by case basis a quick pass is made collecting faulty records and a second pass (maybe with a more rigorouds algorithm) is made of the exceptions. HTH, -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld _______________________________________________ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
