Le lundi 6 juillet 2015 15:17:21 UTC+2, [email protected] a écrit : > > is there any setting for following external links, and crawl outside the > base url? > > I.e. on example.com > <http://www.google.com/url?q=http%3A%2F%2Fexample.com&sa=D&sntz=1&usg=AFQjCNH5iRlnhCg_mk89zURn-g01qQYJEw>, > > you find a link to example1.com, and it continues to crawl example1.com > and find the link example3.com and so on. > > I'm not sure, but perhaps it's ok if you omit allowed_domain parameter.
Nickko -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
