Hi I am not able to crawl this site https://www.ich.org/. Can any one suggest a solution for this. This site does not has robots.txt file. When I try to check robots.txt, site is shown as under construction and returning response status 200. Could it be any reason for issue?
Thanks and Regards Raj Chidara
