Isn't simpler initiate the spider in an isolated process and when it finished inform back the client and the client inform the server?
If you still want to do it your way, maybe scrapyd has the answer. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
