I'mt trying to run spiders created by portia (slyd) using the pipeline elasticsearch plugin [1], and I tried to schedule a spider to run this way:
curl http://localhost:6800/schedule.json -d project=myproject -d spider=myspider -d setting=CLOSESPIDER_ITEMCOUNT=100 -d settings=ITEM_PIPELINE=scrapyelasticsearch.ElasticSearchPipeline -d setting=ELASTICSEARCH_SERVER=localhost -d setting=ELASTICSEARCH_PORT=9200 -d setting=ELASTICSEARCH_INDEX=scrapy -d setting=ELASTICSEARCH_TYPE=items -d setting=ELASTICSEARCH_UNIQ_KEY=url It doesn't work, and the only thing logged is this error message: [Launcher,1900/stderr] > /home/falqon/PORTIA/local/lib/python2.7/site-packages/scrapy/contrib/pipeline/__init__.py:21: > > ScrapyDeprecationWarning: ITEM_PIPELINES defined as a list or a set is > deprecated, switch to a dict > category=ScrapyDeprecationWarning, stacklevel=1) Is there a way to make scrapyd always use the plugin to export data? [1] https://github.com/noplay/scrapy-elasticsearch -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
