Hello All,
I want to extract data in "topics_html" attribute, from "
http://steamcommunity.com/forum/3381077/General/render/0/?start=0&count=10";.
The response is in JSON. I wrote a script like below but I am getting
errors. Please correct the script, I was unable to figure the issue. It is
not even crawling that page.
from scrapy.spider import BaseSpider
from TodayDiscussions.items import TodaydiscussionsItem
import json
import urllib2
class MySpider(BaseSpider):
name = "jsonexample"
allowed_domains = ["steamcommunity.com"]
req =
urllib2.urlopen('http://steamcommunity.com/forum/3381077/General/render/0/?start=0&count=10')
def json_parse(self, response):
jsonresponse = json.loads(response) # also tried
with response.body_as_unicode()
topics_html = jsonresponse["topics_html"]
print topics_html
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
