Hi,
There are several spiders in my one project. I've written my basic global
settings in my project's settings.py and I don't know how to override some
settings for different spider.
Just like spider1 can override settings: ITEM_PIPELINES =
{'myproject.pipelines.UserImagesPipeline':*1*
,'myproject.pipelines.WeibospiderPipeline':300}
while spider2 can stll use the settings: ITEM_PIPELINES =
{'myproject.pipelines.UserImagesPipeline':*None*,'myproject.pipelines.WeibospiderPipeline':300}
in the project's settings.py
I've found the following method in the Core API of the Scrapy
documentation, and I once try to use this method like this:
Solution1:
from scrapy.settings import Settings
class MySpider(BaseSpider):
myset = Settings()
myset.set('ITEM_PIPELINES',{'myproject.pipelines.UserImagesPipeline':
*1*,'myproject.pipelines.WeibospiderPipeline':300},priority='spider')
Solution2:
from scrapy.conf import settings
class MySpider(BaseSpider):
settings.set('ITEM_PIPELINES',{'myproject.pipelines.UserImagesPipeline':1,'myproject.pipelines.pipelines.WeibospiderPipeline':300},priority='spider')
However,these didn't work for me. Besides, Solution2 can change all the
spider's settings, and actually module scrapy.conf is deprecated now.
Is there a way to use different settings for spider1 and spider2 separately?
Thanks!
Bill
<https://lh3.googleusercontent.com/-vYmXOsw5CZ8/VhTXA4dMGOI/AAAAAAAAAAM/Buhab2lEM0g/s1600/aaa.png>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
