Its helped to solve. Thank you. среда, 26 ноября 2014 г., 12:24:03 UTC+2 пользователь Paul Tremberth написал: > > Hello, > > by default, link extractors in Scrapy do URL normalization or URL > canonicalization > https://en.wikipedia.org/wiki/URL_normalization > > You can tell the link extractor not to do that by passing > canonicalize=False > > >>> lx = scrapy.contrib.linkextractors.LinkExtractor() > > >>> lx.extract_links(response)[7] > Link(url=' > http://sflib1.sfpl.org:82/search?%2FdBanks+A%2Fdbanks+a%2F1%2C4%2C41%2CE%2F2exact=&1%2C23%2C=&FF=dbanks+american+trust+company', > > text='Banks American Trust Company', fragment='', nofollow=False) > > >>> lx = scrapy.contrib.linkextractors.LinkExtractor(canonicalize=False) > > >>> lx.extract_links(response)[7] > Link(url=' > http://sflib1.sfpl.org:82/search?/dBanks+A/dbanks+a/1%2C4%2C41%2CE/2exact&FF=dbanks+american+trust+company&1%2C23%2C', > > text='Banks American Trust Company', fragment='', nofollow=False) > > > > > > On Wednesday, November 26, 2014 2:59:00 AM UTC+1, Максимилиан Гомза wrote: >> >> Have a good day. >> >> I am new in Scrapy. I am trying to parse photos from links in page >> http://sfpl.org/index.php?pg=2000028501 >> >> XPATH_WAY = '/html/body/div[2]/div[1]/div[3]/div[2]/div[2]/div[2]/ul/li' >> XPATH_DETALIED = >> ['/html/body/table/tbody/tr[2]/td/table/tbody/tr/td[3]/a[not(@name)]'] >> >> class SplfDetailsSpider(CrawlSpider): >> name = 'splf_details' >> rules = [ >> Rule(LinkExtractor(allow='search', restrict_xpaths=XPATH_WAY)), >> Rule(LinkExtractor(allow='2exact', >> restrict_xpaths=XPATH_DETALIED)), >> Rule(LinkExtractor(allow='frameset'), callback='detail_parse') >> ] >> >> start_urls = ["http://sfpl.org/index.php?pg=2000028501";] >> >> Scrapy without problem parse links from the started page, but in next >> page when he trying to extract links from XPATH_DETALIED. >> There is get a problem. He cuts a part of link, for ecxample: >> >> From page: >> http://sflib1.sfpl.org:82/search/d?SEARCH=Banks+A >> >> First item in xpath's list: >> <a >> href="/search?/dBanks+A/dbanks+a/1%2C4%2C41%2CE/frameset&FF=dbanks+american+national+bank&1%2C1%2C">Banks >> >> American National Bank</a> >> >> Link Extractor cut a link to: >> >> http://sflib1.sfpl.org:82//search?/dBanks+A/dbanks+a/1%2C4%2C41%2CE/frameset&FF=dbanks+american+national+bank >> >> Help please how to resolve this problem. >> >> >> >> >> >>
-- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
