Thanks Travis,
The error messages on the link seem identical to my issue.
"""dmoz_spider.py"""
import scrapy
class DmozSpider(scrapy.Spider):
name = "dmoz"
allowed_domains = ["dmoz.org"]
start_urls = [
"http://www.dmoz.org/Computers/Programming/Languages/Python/Books/";,
"
http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/";
]
def parse(self, response):
filename = response.url.split("/")[-2]
with open(filename, 'wb') as f:
f.write(response.body)
Following the official Scrapy tutorial it seems that the spider class
inherits from scrapy.Spider rather than BaseSpider or CrawlSpider
I am using Twisted 14.0.0 is this OK as I noticed that the Pip download of
latest Scrapy uses Twisted 10.0.0?
Robert
Pastebin link:
<script src="http://pastebin.com/embed_js.php?i=eRULrSDx";></script>
On Friday, December 19, 2014 7:16:39 PM UTC, Travis Leleu wrote:
> Can you post your spider file to pastebin? I've seen this error before
> when people aren't using the correct parent spider to inherit from. See
> http://stackoverflow.com/questions/5264829/why-does-scrapy-throw-an-error-for-me-when-trying-to-spider-and-parse-a-site
>
> <http://www.google.com/url?q=http%3A%2F%2Fstackoverflow.com%2Fquestions%2F5264829%2Fwhy-does-scrapy-throw-an-error-for-me-when-trying-to-spider-and-parse-a-site&sa=D&sntz=1&usg=AFQjCNGCLXzdNDdhL2cUgqpDzJPPSvK8Eg>
>
>
> On Fri, Dec 19, 2014 at 10:51 AM, bobby <[email protected]
> <javascript:>> wrote:
>>
>> Hi,
>>
>> Windows 8.1
>> Anaconda Python 2.7
>> Scrapy 0.24.4
>> Twisted 14.0.0
>>
>> I am working my way though the official scrapy tutorial and tried to run
>> the spider
>>
>>
>>
>> scrapy crawl dmoz
>>
>>
>> This started OK but quickly gave traceback errors in Twisted
>>
>>
>> base.py
>>
>> defer.py
>>
>>
>> and failed at scrapy\spider.py
>>
>>
>> I have made an attached image (aa.png) of the errors.
>>
>>
>> Have tried reinstalling etc but not sure where to try next?
>>
>>
>> Thanks
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scrapy-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> Visit this group at http://groups.google.com/group/scrapy-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
