Also, this is how I got there using scrapy shell..
command list:
'Abegoaria de Baixo' in response.body # returned True, so I knew there was
at least a mention somewhere in the page
hxs.select("//*[contains(.,'Abegoaria de Baixo')]").extract() # an attempt
to get lucky on the first try... didn't work out, produced too much
hxs.select("//a[contains(.,'Abegoaria de Baixo')]").extract() # my next
attempt to get lucky, this one worked out
hxs.select("//a[@target='_top'").extract() # this produced a list of links
about the same length as the list of links your xpath produced on the same
page, figured it was about right
Hope that helps, too
On Tue, Apr 8, 2014 at 8:55 AM, Bill Ebeling <[email protected]> wrote:
> I didn't drill in entirely to the problem, so I don't know what reindeer
> games are being played, but I did have some luck with this:
>
> //a[@target='_top']
>
> I'll try to figure it out when I get some time later on.
>
>
> On Tue, Apr 8, 2014 at 8:37 AM, André Campos <[email protected]> wrote:
>
>> Ok
>>
>> http://www.mapadeportugal.net/concelho.asp?c=1401
>>
>> You can see in the browser that
>> '//p[@class="txtmedioazulb"]//td[@class="txtmedio"]/a' it's available, but
>> not in shell or spider crawling
>>
>> I've tried loading in shell, and the sel.xpath(..) doesn't retrieve
>> nothing, how can i use shell to figure it out ?
>>
>> Thanks!
>>
>> On 08 Apr 2014, at 13:23, Bill Ebeling <[email protected]> wrote:
>>
>> can you post the link to the actual page?
>>
>> Without more information, any suggestions would just be guessing. If you
>> can't, I'd recommend loading the page in scrapy shell and trying to figure
>> it out that way.
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "scrapy-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/scrapy-users/rvq9fGDPRWI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/scrapy-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "scrapy-users" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/scrapy-users/rvq9fGDPRWI/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/scrapy-users.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.
