I want to deploy my scrapy project on scrapyd.
In the setting, I set these values:
FEED_EXPORTERS = {
'jsonlines': 'scrapy.contrib.exporter.JsonLinesItemExporter',
}
FEED_FORMAT = 'jsonlines'
FEED_URI = 'D:\Walid-Project\Tasks\FantasySeriea\output.json'
When I run my spider from cmd, I can see the output.json file contains the
scraped item.
Now, I deployed my project on scrapyd. When I crul the spider and then go
to http://localhost:6800/jobs. I can see the items. but the problem is the
output.json file is not generated.
I checked the log and I saw this
2014-04-30 16:34:51+0300 [scrapy] DEBUG: Overridden settings:
{'NEWSPIDER_MODULE': 'FantasySeriea.spiders', 'SPIDER_MODULES':
['FantasySeriea.spiders'], 'FEED_URI':
'items\\FantasySeriea\\FantasySerieaSpider\\30284e00d06c11e393a9c0f8da6c6a82.jl',
which means that the feed_uri is being overrided.
In the scrapyd.conf I tried to remove the items_dir and I tried to make it
empty and I tried to set the feed_uri explicity like this
FEED_URI = 'D:\Walid-Project\Tasks\FantasySeriea\ourput.json'
but nothing works for me. I still not able to see the output.json file.
Could you help please?
Thanks in advance
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
