I'm playing around with the wget tool and I ran into this website that I don't believe the "-e robots=off" works. http://www.quickmba.com/ any idea why?
I've tried a few combinations and I keep on getting this message in the response. We're sorry, but the way that you have attempted to access this site is not permitted by the QuickMBA User Agreement. Please contact us via the site contact form if you have any questions or believe that you have received this message in error. Any ideas why?