I can't believe it was that simple :D. Thanks! Also now I get this error: exceptions.IndexError: list index out of range
That's because 'Plot Size (m²)' field is not present in some properties and that's fine, but why is scrapy by default just ignoring fields that can be scraped and move automatically on the next page? Is there a way to scrape the rest of the fields on the page and if 'Plot Size (m²)' field is not available print NULL or 0? субота, 22. август 2015. 16.16.33 UTC+2, Paul Tremberth је написао/ла: > > Try using Unicode string parameter for xpath() > response.xpath(u'//span[text()="Plot Size (m²)"]/preceding::span/text()') > Le 22 août 2015 15:00, "Mario" <[email protected] <javascript:>> a > écrit : > >> I'm having issues with getting some values from this page: >> >> >> http://www.remax-malta.com/Maisonette-For-Sale-St-Pauls-Bay-North_240041024-145 >> >> To be more specific here's a picture of what I can(colored green) and >> can't(colored red) scrape: >> >> http://i.imgur.com/xT6wTtl.png >> >> Example of writing xpath for Total rooms is: >> >> response.xpath('//span[text()="Total Rooms:"]/preceding::span/text()'). >> extract()[-1] >> >> This prints: u'3' which is value I'm after. >> >> But when I try to write xpath for Plot Size (m²) like this one: >> >> response.xpath('//span[text()="Plot Size (m²)"]/preceding::span/text()'). >> extract()[-1] >> >> >> I get this error: >> >> ValueError: All strings must be XML compatible: Unicode or ASCII, no NULL >> bytes or control characters >> >> >> I know I get this because of '²' character. Can somebody help me out >> with writing proper xpath? Or maybe there's another way of getting value >> from xpath? >> >> -- >> You received this message because you are subscribed to the Google Groups >> "scrapy-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at http://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
