On Fri, Aug 26, 2016 at 12:23:00AM -0700, Riad S. Wahby wrote: > Georgi Guninski <gunin...@guninski.com> wrote: > > Which parts of the cpunks.org web archive are indexed by google? > > No idea. I had assumed that without a robots.txt Google would just > slurp up everything, but perhaps that's not true. Anyhow, I added > one just in case (though you'll have to clear your browser's cache > because the blanket rewrite rule uses 301s). >
I suppose the redirected robots.txt is not a problem and it used to work.
