On Monday, October 8, 2018 at 12:18:27 AM UTC-4, Hassan Schroeder wrote: > > On Fri, Oct 5, 2018 at 11:04 PM, fugee ohu <fuge...@gmail.com > <javascript:>> wrote: > > I see bots crawling devise pages in my log: > > > Should I disallow those paths in robots.txt ? > > 1) why? > > 2) do your logs show these bots *reading* robots.txt? > > -- > Hassan Schroeder ------------------------ hassan.s...@gmail.com > <javascript:> > twitter: @hassan > Consulting Availability : Silicon Valley or remote >
Yes -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to rubyonrails-talk+unsubscr...@googlegroups.com. To post to this group, send email to rubyonrails-talk@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/1fe76e02-c222-450b-8eca-16e4df491f18%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
