Sayth,
Out of curiousity, what do you like about jquery that isn't available in
scrapy's .cssselect() method? I had thought they were pretty comparable:
res = d('nomination')
With scrapy, would be something like:
res = selector.cssselect('.nomination') // assuming nomination is a
class you want to select on
I'm just curious to learn what you like more about pyquery, and if you were
aware of the .cssselect() method.
On Tue, Apr 19, 2016 at 5:02 AM, Sayth Renshaw <[email protected]>
wrote:
>
> Ah cool so I had started writing a script with pyquery and just loved how
> easy its selectors were. This is so easy compared to Xpath and compact this
> below is main excert:
>
> for items in fileResult:
> # d = pq(*filename=items*)
> d = pq(response.text)
> res = d('nomination')
> attrs = ('id', 'horse')
> data = [[res.eq(i).attr(x) for x in attrs] for i in range(len(res))]
>
> so essentially my only change is taking the filename attribute and
> replacing it with the httpresponse and I will get to use the rest of
> scrapy's plumbing for free?
>
> Sounds like a good deal.
>
> Sayth
>
> On Tue, 19 Apr 2016 at 21:33 Paul Tremberth <[email protected]>
> wrote:
>
>> Hello,
>>
>> sure you can use PyQuery.
>> Callbacks get passed HTTP responses, which are `scrapy.http.HtmlResponse`
>> most of the time,
>> so you do something like
>>
>> from pyquery import PyQuery as pq
>> ...
>>
>> class MySpider(Spider):
>> ...
>>
>> def parse_page(self, response):
>> d = pq(response.body_as_unicode())
>> # or if you're using Scrapy 1.1(rc3)
>> # d = pq(response.text)
>>
>> Hope this helps.
>>
>> Paul.
>>
>> On Tuesday, April 19, 2016 at 1:18:03 PM UTC+2, Sayth Renshaw wrote:
>>>
>>> Hi
>>>
>>> Is it possible to use Pyquery in scrapy so i can get the easy jquery
>>> selector syntax but then use the scrapy pipelines to manage data?
>>>
>>> The pyquery options made selecting so easy just wanted to see if i could
>>> get the benefits of both the scrapy framework and the easy selectors.
>>>
>>> Cheers
>>>
>>> Sayth
>>>
>>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "scrapy-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/scrapy-users/8Pi0Qw1pmok/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at https://groups.google.com/group/scrapy-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "scrapy-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To post to this group, send email to [email protected].
> Visit this group at https://groups.google.com/group/scrapy-users.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
