It looks like it's updating the value via javascript. You'll either need to mimic the ajax call (if it's retrieving data from the server based on current conversion rate), or recreate the logic in your crawler.
Scrapy doesn't execute javascript. On Wed, Nov 11, 2015 at 10:30 AM, Mario <[email protected]> wrote: > Ok so a little update, it seems like I used form_data instead of > formdata. Now the spider runs normally but it still scrapes prices that are > in $, so it means that it didn't change at all :( > > > среда, 11. новембар 2015. 15.57.04 UTC+1, Mario је написао/ла: >> >> Hi, >> >> So I have issues with integrating forms into spider. The spider itself >> its really simple, here’s the code: >> >> import scrapyfrom scrapy.http import FormRequest, Request >> class YellowkornerSpider(scrapy.Spider): >> name = "yellowkorner" >> allowed_domains = ["yellowkorner.com"] >> start_urls = ('http://www.yellowkorner.com/photos/index.aspx') >> >> def parse(self, response): >> yield FormRequest.from_response(response, formname='aspnetForm', >> form_data={'ctl00$languageSelection$d...': 'GB'}, >> callback=self.parse_photographs) >> >> def parse_photographs(self, response): >> prices = response.xpath('//*[@class="price"]/span/text()').extract() >> print prices >> >> The error that I’m getting right now is this: >> >> TypeError: __init__() got an unexpected keyword argument 'form_data' >> >> Also what I’m trying to accomplish is to scrape prices from this page >> <http://www.yellowkorner.com/photos/index.aspx>. The issue is that I >> want prices in UK not in US, so instead of getting prices in $ I will need >> prices in £. >> >> I know issue is in yeilding form part but I don’t know how to tackle >> this. Here’s the part where countries are changed: >> >> [image: ss] >> >> And here’s form name, I hope I got it right: >> >> [image: ss] >> >> Here’s the page <http://stackoverflow.com/a/11219623> where I got form >> part implemented. >> >> Any help is appreciated I’m trying to get this right by days now… >> >> > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
