Take a look at portiacrawl <https://github.com/scrapinghub/portia/blob/master/slybot/bin/portiacrawl> command - it basically sets up some environment variables and command line args and then calls scrapy. The main difference to a typical spider is the use of a custom spider manager, so that scrapy can work with spiders defined as json.
You should be able to use scrapy extensions with portia. I am not sure about scrapyd, but as long as you can define the same environment and scrapy is invoked with the same args it should work. On 6 June 2014 18:51, christoph <[email protected]> wrote: > Hi, > > for a project I need to run Portia on a specific server and deploy the > project to another server to perform the actual crawl and control this > crawl. > Marcos from Scrapinghub pointed out that I have to run crawl projects > created with Portia with the "portiacrawl" command. Thanks for the helpful > information. > > Since Portia project aren't the same as Scrapy projects, does it mean I > cannot run tools like scrapyd or scrapy-redis with Portia projects? > > Why is there a difference in Portia projects and Scrapy projects? The > Portia readme says: "Portia spiders are ultimately Scrapy spiders." > > Thank you, > Christoph > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
