Annie, The problem you're experiencing is because the map is loaded via javascript. Scrapy has no javascript capabilities; for that, you'll need something like PhantomJS or CasperJS with Selenium.
On Thu, Oct 9, 2014 at 9:28 AM, Annie Kim <[email protected]> wrote: > Hi everyone, > > I am crawling this website: > http://www.osvita.com.ua/ua/schools/kharkov/sh-2/ and want to get > latitude and longitude of the place through the map. > The structure is like this. What I need to do is to get the highlighted > href link and get lat/long from that link. > > > <https://lh5.googleusercontent.com/-CbnGHG4p8yo/VDa3Qd4dwjI/AAAAAAAAALM/YgQ8zPt0gn4/s1600/Screen%2BShot%2B2014-10-09%2Bat%2B5.19.04%2BPM.png> > > > When I use xpath to extract '//*[@id="YMapsID"]', I got [<Selector > xpath='//*[@id="YMapsID"]' data=u'<div id="YMapsID" > style="width:668px;hei'>]. > But when I further extract '//*[@id="YMapsID"]/div', I got nothing. > I was wonder if the problem is because the map needs more time to be > loaded.Do we have a setting to extend the time needed for loading a url? > > Or any other ideas? > > Thanks, > Annie > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
