Yes I'm building a webapp, to see crawling jobs scheduled by crontab. Once I've checked everything ran right, I want to delete these jobs much for presentation purpose than disk space (event if I also care about it). Let's say you have 5 spiders running each night during a year, I will become quite uncomfortable to display them all (even if can I can sort them by date) but most important I don't need to keep past jobs results.
By the way, I've thought about the middleware we previously talked about : 1. I can definetly delete .log and .jl files since I know where they are stored 2. But when calling the "listjobs.json" (http://scrapyd.readthedocs.org/en/latest/api.html#listjobs-json) method, I will still be retrieving under the "finished" items those I want to get away from this list So the question could be renamed "how to remove finished jobs from scrapyd" By the way I've created a git repository of my projet so you can see what I'm working on : https://github.com/hugsbrugs/scrapyd-webapp Let's continue the talk ... ;) -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
