We had to disable rules 960009 and 960015 also as they were causing a lot of false positives for legitimate website visitors. I think a lot of valid/different web browsers don't conform to Accept and User agent headers properly.
Wes -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of [email protected] Sent: May 15, 2015 8:52 AM To: [email protected] Subject: Owasp-modsecurity-core-rule-set Digest, Vol 73, Issue 5 Send Owasp-modsecurity-core-rule-set mailing list submissions to [email protected] To subscribe or unsubscribe via the World Wide Web, visit https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set or, via email, send a message with subject or body 'help' to [email protected] You can reach the person managing the list at [email protected] When replying, please edit your Subject line so it is more specific than "Re: Contents of Owasp-modsecurity-core-rule-set digest..." Today's Topics: 1. Re: Rule 960009 generates false positives from my own server IP (Guilherme Y) 2. Re: Rule 960009 generates false positives from my own server IP (Guilherme Y) ---------------------------------------------------------------------- Message: 1 Date: Fri, 15 May 2015 09:15:43 -0300 From: Guilherme Y <[email protected]> To: Barry Pollard <[email protected]>, "[email protected]" <[email protected]> Subject: Re: [Owasp-modsecurity-core-rule-set] Rule 960009 generates false positives from my own server IP Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" Many thanks Barry!I appreciate your sharing of your own experience, pretty much exactly the same we face here.Yes, one of our apps generates too many useless connections to Apache. We will work on a new version optimizing this but it will take a while, perhaps a couple of months, and in the meantime we will have to work around with an exception rule.Our mod_security install is standard, e.g. we don't have any exceptions configured yet.So I am not sure as where to put that rule.OWASP is in this path: /usr/local/apache/conf/modsec_vendor_configs/OWASPAnd there's only conf file there: modsecurity_crs_10_setup.confIs it OK to place the @ipmatch exception rule in a file namedmodsecurity_crs_15_localrules.confunder this same folder? Tks a lot!All the best!Luiz > From: [email protected] > To: [email protected]; > [email protected] > Subject: RE: [Owasp-modsecurity-core-rule-set] Rule 960009 generates > false positives from my own server IP > Date: Wed, 13 May 2015 10:07:24 +0100 > > That would work though I personally prefer to use @ipMatch which also allows you to match multiple IPs: > > SecRule REMOTE_ADDR "@ipMatch XXX.YYY.Z.WWW XXX.YYY.Z.VVVV" "id:1000,phase:2,t:none,pass,nolog,ctl:ruleRemoveById=960009" > > A few other things to note: > 1) 960009 is a phase 2 rule and you have specified phase 1 in your rule below. I'm pretty sure that rules cannot see other rules outside of their phase so it needs to be in the same phase. So I've updated this to phase 2 in my version above. > 2) Your rule must have it's own id (as of ModSecurity 2.7) so I've > given it an id of 1000. Make sure this doesn't clash with any existing > rule you have. Principal is anything less than 99,999 are your own > use. See here for more info: > https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#id > 3) You could disable ModSecurity completely for internal addresses with a ctl:ruleEngine=off rule. Personally I don't like that as I want them on for testing purposes so we're not caught out with a rule that only runs in production so think just disabling the specific rules is a better option. > > However I think it would be good to understand why you need to do this at all. > > Rule 960009 checks for requests missing a user agent. All decent browsers send a user agent hence why this rule is in place as the lack of a user agent is probably due to some script or such like, so it is a good rule. > > So why are you getting so many requests from internal addresses without a user agent and should you fix that rather than add the above rule to get around that? > > I can think of two reasons: > > 1) Monitoring scripts or tools which are not quite as compliant as web browsers. So perhaps change these to also report the user agent as that could be useful information to know (though with third party tools that might not always be possible). > > 2) Needlessly connecting to Apache. We use Apache to server up static pages and also in proxy mode to pass dynamic requests on to an application server. After installing mod security and noticing similar to you, we investigated and discovered some of our dynamic applications were needlessly making calls to Apache, just to connect back to the app server again. It was actually better to change them to connect directly to the app server, which resolved the issue, as well as saving network traffic and a (admittedly small amount of) Apache load. > > Thanks, > Barry > > ________________________________ > > From: [email protected] > > To: [email protected] > > Date: Tue, 12 May 2015 20:34:22 -0300 > > Subject: [Owasp-modsecurity-core-rule-set] Rule 960009 generates > > false positives from my own server IP > > > > Hi! > > > > I don't know if anyone experiences the same issue as us here, but I > > suppose at least this might contribute to all. > > > > I installed OWASP rules on a Centos running 2 Joomla sites with > > nearly > > 5,000 unique visitors a day. > > > > I was fortunate enough to identify and disable 12 rules that > > delivered a bunch of false positives (one of them locked down the > > server when one of us in the team submitted a security scan from CSF/LFD...). > > > > So, now it is running fine but one rule still delivers near 1,000 > > false positives a day and oddly enough having our own server IP as source! > > > > And severity level for ALL of the hits are NOTICE. So, this is not > > so much troublesome, except for the extra load on the server and the > > log size. I rotate it automatically everynight but it comes out at > > nearly > > 0,3 GB as standard size. > > > > So, what I am trying to do but don't know exactly how is to implent > > something like this in a file named modsecurity_crs_15_localrules.conf: > > > > SecRule REMOTE_ADDR "@streq XXX.YYY.Z.WWW" > >> "phase:1,t:none,pass,nolog,ctl:ruleRemoveById=960009" > > > > where XXX.YYY.Z.WWW is my server's IP address. > > > > Does anyone know if this is correct and if it can actually work to > > keep my server out of this rule execution? > > > > Tks a lot! > > > > All the best! > > Luiz Guilherme > > > > _______________________________________________ > > Owasp-modsecurity-core-rule-set mailing list > > [email protected] > > https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule > > -set > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/attachment s/20150515/57316fef/attachment-0001.html> ------------------------------ Message: 2 Date: Fri, 15 May 2015 11:51:13 -0300 From: Guilherme Y <[email protected]> To: Barry Pollard <[email protected]>, "[email protected]" <[email protected]> Subject: Re: [Owasp-modsecurity-core-rule-set] Rule 960009 generates false positives from my own server IP Message-ID: <[email protected]> Content-Type: text/plain; charset="iso-8859-1" Hi Barry!Well, it didn't work...This is what I did:My mod_security conf and rules files are set up like this: modsecurity_crs_15_customrules.conf file has only this line: SecRule REMOTE_ADDR "@ipMatch 999.99.99.99" "id:1000,phase:2,t:none,pass,nolog,ctl:ruleRemoveById=960009" I restarted Apache, but rule 960009 still process that IP as source. What did I do wrong? There is also a file named /usr/local/apache/conf/modsec2.whitelist.conf with just this on it: # ConfigServer ModSecurity whitelist file<LocationMatch .*></LocationMatch> I'm confused, don't know in which files this directive should go or what else should I have done.Tks!All the best!Luiz > From: [email protected] > To: [email protected]; > [email protected] > Subject: RE: [Owasp-modsecurity-core-rule-set] Rule 960009 generates > false positives from my own server IP > Date: Wed, 13 May 2015 10:07:24 +0100 > > That would work though I personally prefer to use @ipMatch which also allows you to match multiple IPs: > > SecRule REMOTE_ADDR "@ipMatch XXX.YYY.Z.WWW XXX.YYY.Z.VVVV" "id:1000,phase:2,t:none,pass,nolog,ctl:ruleRemoveById=960009" > > A few other things to note: > 1) 960009 is a phase 2 rule and you have specified phase 1 in your rule below. I'm pretty sure that rules cannot see other rules outside of their phase so it needs to be in the same phase. So I've updated this to phase 2 in my version above. > 2) Your rule must have it's own id (as of ModSecurity 2.7) so I've > given it an id of 1000. Make sure this doesn't clash with any existing > rule you have. Principal is anything less than 99,999 are your own > use. See here for more info: > https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#id > 3) You could disable ModSecurity completely for internal addresses with a ctl:ruleEngine=off rule. Personally I don't like that as I want them on for testing purposes so we're not caught out with a rule that only runs in production so think just disabling the specific rules is a better option. > > However I think it would be good to understand why you need to do this at all. > > Rule 960009 checks for requests missing a user agent. All decent browsers send a user agent hence why this rule is in place as the lack of a user agent is probably due to some script or such like, so it is a good rule. > > So why are you getting so many requests from internal addresses without a user agent and should you fix that rather than add the above rule to get around that? > > I can think of two reasons: > > 1) Monitoring scripts or tools which are not quite as compliant as web browsers. So perhaps change these to also report the user agent as that could be useful information to know (though with third party tools that might not always be possible). > > 2) Needlessly connecting to Apache. We use Apache to server up static pages and also in proxy mode to pass dynamic requests on to an application server. After installing mod security and noticing similar to you, we investigated and discovered some of our dynamic applications were needlessly making calls to Apache, just to connect back to the app server again. It was actually better to change them to connect directly to the app server, which resolved the issue, as well as saving network traffic and a (admittedly small amount of) Apache load. > > Thanks, > Barry > > ________________________________ > > From: [email protected] > > To: [email protected] > > Date: Tue, 12 May 2015 20:34:22 -0300 > > Subject: [Owasp-modsecurity-core-rule-set] Rule 960009 generates > > false positives from my own server IP > > > > Hi! > > > > I don't know if anyone experiences the same issue as us here, but I > > suppose at least this might contribute to all. > > > > I installed OWASP rules on a Centos running 2 Joomla sites with > > nearly > > 5,000 unique visitors a day. > > > > I was fortunate enough to identify and disable 12 rules that > > delivered a bunch of false positives (one of them locked down the > > server when one of us in the team submitted a security scan from CSF/LFD...). > > > > So, now it is running fine but one rule still delivers near 1,000 > > false positives a day and oddly enough having our own server IP as source! > > > > And severity level for ALL of the hits are NOTICE. So, this is not > > so much troublesome, except for the extra load on the server and the > > log size. I rotate it automatically everynight but it comes out at > > nearly > > 0,3 GB as standard size. > > > > So, what I am trying to do but don't know exactly how is to implent > > something like this in a file named modsecurity_crs_15_localrules.conf: > > > > SecRule REMOTE_ADDR "@streq XXX.YYY.Z.WWW" > >> "phase:1,t:none,pass,nolog,ctl:ruleRemoveById=960009" > > > > where XXX.YYY.Z.WWW is my server's IP address. > > > > Does anyone know if this is correct and if it can actually work to > > keep my server out of this rule execution? > > > > Tks a lot! > > > > All the best! > > Luiz Guilherme > > > > _______________________________________________ > > Owasp-modsecurity-core-rule-set mailing list > > [email protected] > > https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule > > -set > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/attachment s/20150515/96cb193b/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: files.png Type: image/png Size: 14858 bytes Desc: not available URL: <http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/attachment s/20150515/96cb193b/attachment.png> ------------------------------ _______________________________________________ Owasp-modsecurity-core-rule-set mailing list [email protected] https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set End of Owasp-modsecurity-core-rule-set Digest, Vol 73, Issue 5 ************************************************************** _______________________________________________ Owasp-modsecurity-core-rule-set mailing list [email protected] https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
