On 9/20/07, John Melesky <[EMAIL PROTECTED]> wrote: > I don't have this problem (assuming your problem is that no HTML is > served to the curl request). What is your config? >
Just to be clear, the bot came and made a HEAD request to a number of pages. A HEAD request isn't supposed to serve html, just the headers. After the HEAD requests I didn't see the bot for days, while the webmaster tools said I had very much been crawled successfully, yet none of my new content was in their tool view of the index. I presume it's doing the HEAD request to conserve bandwidth and decide if it should make a real GET request later. It also is likely sending some kind of If-Modified-Since or the like, and my site might be sending the wrong answer, which is why I ask about configuration. But I'm wondering specifically if turning on upstream caching would cause django to have more to say, etags, etc. Or I'm just way too impatient. I suspect I just need to wait for awhile. -- Darrin --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
