Thanks, Adam. According to the documentation Genieo provides it sounds like their crapware is making additional requests on behalf of the user:
Genieo is designed to automatically retrieve and filter information items > from across the web, based on the user's specific individual interests, and > display them on a personal Homepage. > Genieo studies the user's preferred individual interests and sources at a > high resolution, by analyzing their browsing routine. Our desktop > configuration runs on the user's compuer to increase privacy. Genieo then > continuously explores the internet for specific information items which are > mostly related to these interests, and presents them on the personal > Homepage. > The Genieo homepage is styled as a newspaper front page with selected > items, consisting of title, short snippet, media, link to article, and > share button My initial request to them for help indicated that their software DID respect robots.txt, which is why I went this route. I am also looking at a Rack-based solution to simply redirect any requests with that user agent. Chris On Thu, Mar 13, 2014 at 11:24 AM, Adam Grant <[email protected]> wrote: > Question: Does the crapware modify your user's browser request, or is it > making a second request unbeknownst to your user? > > Crapware doesn't usually respect robots.txt. You might need to put a > middleware in your Rails app to filter out that user agent, or something at > the Apache/Nginx level. Not ideal, but at least it won't trigger a routing > error. > > Regards, > - Adam > > > On Thu, Mar 13, 2014 at 11:17 AM, Chris McCann <[email protected]>wrote: > >> I've had a rash of exception notifications come through from one of my >> Rails apps lately. >> >> A ActionController::MethodNotAllowed occurred in application#index: >>> Only put and delete requests are allowed. >> >> >> The culprit appears to be this: >> >> HTTP_USER_AGENT : Mozilla/5.0 (compatible; Genieo/1.0 >> http://www.genieo.com/webfilter.html) >> >> You can click the link to see what Genieo is (tl;dr: widely panned as >> crapware). It looks to me like one of my users has this crapware on their >> computer, had it running when they interacted with my app at a URL that >> should take a PUT, and now Genieo keeps hitting that link with a GET, >> triggering the error. >> >> Looking for ways to prevent this it seemed that putting a disallow for >> Genieo in robots.txt would solve it. Here's my robots.txt file: >> >> # See http://www.robotstxt.org/wc/norobots.html for documentation on how to >> use the robots.txt file >> >> # >> >> # To ban all spiders from the entire site uncomment the next two lines: >> >> User-Agent: Genieo >> >> Disallow: / >> >> >> >> After redeploying I'm still getting the hits but don't understand why. I've >> put a request into their support to figure this out, but I'm curious if >> anyone else here has dealt with a problem like this in a similar fashion. >> >> >> Cheers, >> >> >> >> Chris >> >> -- >> -- >> SD Ruby mailing list >> [email protected] >> http://groups.google.com/group/sdruby >> --- >> You received this message because you are subscribed to the Google Groups >> "SD Ruby" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > -- > SD Ruby mailing list > [email protected] > http://groups.google.com/group/sdruby > --- > You received this message because you are subscribed to a topic in the > Google Groups "SD Ruby" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/sdruby/v8YBg5uRDj0/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- -- SD Ruby mailing list [email protected] http://groups.google.com/group/sdruby --- You received this message because you are subscribed to the Google Groups "SD Ruby" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
