Does it crawl other sites with Arabic characters? Remi
On Tuesday, January 31, 2012, mina <tahereganji...@gmail.com> wrote: > i crawl a site with nutch 1.4, i understand that nutch dosen't crawl all > links in this site. i have no filter and no limit rule to crawling. for > example nutch never crawl this link: > > > http://www.irna.ir/News/30786427/سوء-استفاده-از-نام-كمیته-امداد-برای-جمع-آوری-رای-در-مناطق-محروم/سياسي/ > > if i give this link to nutch to crawl, nutch never crawl this link. > how i can crawl this link? > > -- > View this message in context: http://lucene.472066.n3.nabble.com/why-nutch-dosen-t-crawl-all-links-tp3702031p3702031.html > Sent from the Nutch - User mailing list archive at Nabble.com. >
