Hi Travis, thanks for the reply.
I messed something up with post so I deleted it and created new post.
I've used forms before on somewhat similiar issue that I was having but
then there were only 3 items that were populated. So basically it looked
like this:
```python
form_data = {"school_name": '', "school_type_id": school_type_id,
"state_id": state_id}
request = scrapy.FormRequest(url, callback=self.parse_schools,
formdata=form_data)
yield request
```
And that worked flawlessly. As for issue that I'm facing now I think this
can be done via forms but I'm newbie at this so I don't know how to...
среда, 11. новембар 2015. 19.56.15 UTC+1, Travis Leleu је написао/ла:
>
> It looks like it's updating the value via javascript. You'll either need
> to mimic the ajax call (if it's retrieving data from the server based on
> current conversion rate), or recreate the logic in your crawler.
>
> Scrapy doesn't execute javascript.
>
> On Wed, Nov 11, 2015 at 10:30 AM, Mario <[email protected]
> <javascript:>> wrote:
>
>> Ok so a little update, it seems like I used form_data instead of
>> formdata. Now the spider runs normally but it still scrapes prices that are
>> in $, so it means that it didn't change at all :(
>>
>>
>> среда, 11. новембар 2015. 15.57.04 UTC+1, Mario је написао/ла:
>>>
>>> Hi,
>>>
>>> So I have issues with integrating forms into spider. The spider itself
>>> its really simple, here’s the code:
>>>
>>> import scrapyfrom scrapy.http import FormRequest, Request
>>> class YellowkornerSpider(scrapy.Spider):
>>> name = "yellowkorner"
>>> allowed_domains = ["yellowkorner.com"]
>>> start_urls = ('http://www.yellowkorner.com/photos/index.aspx')
>>>
>>> def parse(self, response):
>>> yield FormRequest.from_response(response, formname='aspnetForm',
>>> form_data={'ctl00$languageSelection$d...': 'GB'},
>>> callback=self.parse_photographs)
>>>
>>> def parse_photographs(self, response):
>>> prices = response.xpath('//*[@class="price"]/span/text()').extract()
>>> print prices
>>>
>>> The error that I’m getting right now is this:
>>>
>>> TypeError: __init__() got an unexpected keyword argument 'form_data'
>>>
>>> Also what I’m trying to accomplish is to scrape prices from this page
>>> <http://www.yellowkorner.com/photos/index.aspx>. The issue is that I
>>> want prices in UK not in US, so instead of getting prices in $ I will need
>>> prices in £.
>>>
>>> I know issue is in yeilding form part but I don’t know how to tackle
>>> this. Here’s the part where countries are changed:
>>>
>>> [image: ss]
>>>
>>> And here’s form name, I hope I got it right:
>>>
>>> [image: ss]
>>>
>>> Here’s the page <http://stackoverflow.com/a/11219623> where I got form
>>> part implemented.
>>>
>>> Any help is appreciated I’m trying to get this right by days now…
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "scrapy-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> Visit this group at http://groups.google.com/group/scrapy-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
