Hi Frédéric, what do you mean by "smart refresh crawling"? scrapylib has the DeltaFetch spider middleware https://github.com/scrapinghub/scrapylib/blob/master/scrapylib/deltafetch.py
Paul. On Wednesday, July 16, 2014 10:15:11 AM UTC+2, Magikmeuh wrote: > > Hello everyone, > > Does scrapy have a smart refresh crawling algorithm ? > > I don't see any trace of it in the documentation or on this googlegroup; > > Does someone have already implemented it ? > > Thanks > > > -- > Frédéric Passaniti > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
