"Beyond Control Inc." <[EMAIL PROTECTED]> writes: > Unfortunatly, if I start it now it *will* work for 10 hours, then die.
How many pages has been fetched at this point? 6 each hour for 12 hours are 72, right? Does it fail quickly if you don't wait for 10 minutes between each fetch? Regards, Gisle > This is why I find this to be so weird, since I look like IE5, and I always > create a new UserAgent object so that I won't get cookied or anything like > that... > > ----- Original Message ----- > From: "William R Ward" <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Wednesday, November 28, 2001 11:55 PM > Subject: Re: 500 Read error > > > > [EMAIL PROTECTED] (Beyond Control Inc.) writes: > > > I've written a small script that fetches a page every 10 minutes. > > > > > > After 12-14 hours, it suddenly stops fetching the page, say there's a > > > read error (500) > > > > Is this something that works once you start it up again? Or did it > > work for a while and now you can't get it to work anymore? If the > > latter, it could be that the web administrators noticed your script > > and blocked it. Some web site owners don't like people slurping their > > data when the ads or copyright notices might not be slurped as well.
