Ah cool so I had started writing a script with pyquery and just loved how
easy its selectors were. This is so easy compared to Xpath and compact this
below is main excert:
for items in fileResult:
# d = pq(*filename=items*)
d = pq(response.text)
res = d('nomination')
attrs = ('id', 'horse')
data = [[res.eq(i).attr(x) for x in attrs] for i in range(len(res))]
so essentially my only change is taking the filename attribute and
replacing it with the httpresponse and I will get to use the rest of
scrapy's plumbing for free?
Sounds like a good deal.
Sayth
On Tue, 19 Apr 2016 at 21:33 Paul Tremberth <[email protected]>
wrote:
> Hello,
>
> sure you can use PyQuery.
> Callbacks get passed HTTP responses, which are `scrapy.http.HtmlResponse`
> most of the time,
> so you do something like
>
> from pyquery import PyQuery as pq
> ...
>
> class MySpider(Spider):
> ...
>
> def parse_page(self, response):
> d = pq(response.body_as_unicode())
> # or if you're using Scrapy 1.1(rc3)
> # d = pq(response.text)
>
> Hope this helps.
>
> Paul.
>
> On Tuesday, April 19, 2016 at 1:18:03 PM UTC+2, Sayth Renshaw wrote:
>>
>> Hi
>>
>> Is it possible to use Pyquery in scrapy so i can get the easy jquery
>> selector syntax but then use the scrapy pipelines to manage data?
>>
>> The pyquery options made selecting so easy just wanted to see if i could
>> get the benefits of both the scrapy framework and the easy selectors.
>>
>> Cheers
>>
>> Sayth
>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "scrapy-users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/scrapy-users/8Pi0Qw1pmok/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> [email protected].
> To post to this group, send email to [email protected].
> Visit this group at https://groups.google.com/group/scrapy-users.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
