I wanted to say thanks to those of you who have sent in some stats! I see that some users are already using Christian Bockermann’s updated jwall-tools.jar file to gather stats which is cool.
We still needs more stats sent in though, so if you have a minute, please send them in. Thanks again for your help. -Ryan On 10/4/10 4:49 PM, "Ryan Barnett" <[email protected]> wrote: Greetings everyone, I am asking for some assistance from the ModSecurity user-base. I am working on a project and need to gather some high level statistics of ModSecurity event data. Our long-term goal is to eventually have a statistical reporting utility that ModSec users can run through cron or something to report data in semi-realtime batches. We can then post this data onto the ModSecurity.org website. This data would only be an anonymous count of the ModSecurity/CRS Event message data (SQL Injection Attack, Invalid character in request, etc....). It will not include any sensitive data identifying web sites, users or their data. In the short-term, what I am asking for you all to do to help with this initial effort, is to simply run the following command pipeline against their saved ModSecurity audit log data. The audit log can be either serial or concurrent. All you need to do is to traverse into your logs directory (example - /usr/local/apache/logs). This directory needs to be the directory that holds either the serial modsec_audit.log file or where the concurrent SecDataDir directory is located. Next, execute the following command (you may need to use sudo or something if you do not have the correct permissions to view the file) - $egrep -R '^Message\:' . | awk -F' \\[msg ' '{ print $2 }' | awk -F'"' '{print $2}' | egrep -v '^(Inbound Anomaly Score|$)' | sort | uniq -c | sort -nr This command will extract out the ModSecurity Message lines from the audit_log data and then sort and unique the ModSecurity event message “msg” data from all of the events. The output format should be similar to this - 3809 Rogue web site crawler 3809 Request Indicates a Security Scanner Scanned the Site 3808 Request Missing an Accept Header 795 Cross-site Scripting (XSS) Attack 538 Detects very basic XSS probings 478 Detects possibly malicious html elements including some attributes 474 Detects obfuscated script tags and XML wrapped HTML 434 Detects specific directory and path traversal 421 XSS Attack Detected 396 IE XSS Filters - Attack Detected 359 Remote File Access Attempt 324 Possible XSS Attack Detected - HTML Tag Handler 318 Detects basic directory traversal 274 Detects etc/passwd inclusion attempts 155 Host header is a numeric IP address 155 finds html breaking injections including whitespace attacks 145 Detects JavaScript object properties and methods 116 Detects basic SQL authentication bypass attempts 2/3 112 Detects common XSS concatenation patterns 1/2 96 Detects classic SQL injection probings 2/2 94 Detects self-executing JavaScript functions 84 Detects possible includes and typical script methods 83 Invalid character in request 69 Restricted Character Anomaly Detection Alert - Repetative Non-Word Characters 54 System Command Injection 32 Detects common comment types 31 Detects possible includes, VBSCript/JScript encodeed and packed functions 30 Comment Evasion Attempt 27 System Command Access 25 More than 3 times special encode Error 25 Detects JavaScript location/document property access and window access obfuscation 18 GET or HEAD requests with bodies 18 Detects basic SQL authentication bypass attempts 3/3 14 Restricted Character Anomaly Detection Alert - Total # of special characters exceeded 13 Detects JavaScript language constructs 12 HTTP protocol version is not allowed by policy 10 Detects data: URL injections, VBS injections and common URI schemes 9 Detects classic SQL injection probings 1/2 8 Input Validation Error 8 Detects code injection attempts 3/3 7 Detects chained SQL injection attempts 1/2 6 Invalid HTTP Request Line 6 finds attribute breaking injections including whitespace attacks 5 Method is not allowed by policy 4 SQL Injection Attack 4 Detects obfuscated JavaScript script injections 4 Detects halfwidth/fullwidth encoded unicode HTML breaking attempts 4 Detects common function declarations and special JS operators 4 Detects code injection attempts 2/3 3 Request Missing a Host Header 3 Detects MySQL comment-/space-obfuscated injections 3 Detects basic obfuscated JavaScript script injections 2 Unicode Full/Half Width Abuse Attack Attempt 2 Remote File Inclusion Attack 2 Hexadecimal Charcode Pattern Found 2 finds unquoted attribute breaking injections 2 Detects the IE octal, hex and unicode entities 2 Detects nullbytes and other dangerous characters 2 Detects JavaScript string properties and methods 2 Detects basic SQL authentication bypass attempts 1/3 1 URL Encoding Abuse Attack Attempt 1 Request Missing a User Agent Header 1 Possible RegEx DoS Payload 1 Detects basic XSS DoS attempts 1 Blind SQL Injection Attack 1 Backdoor access Once you have run this against your saved logs, please send them to me in email to the following email address – [email protected]. If you could, also please specify the following in the email body - 1. How many websites ModSecurity is monitoring, and 2. The date range of the data included in the stats report (we are looking for all data from 2010). Thanks for your help in this research effort. Ryan Barnett ModSecurity Community Manager OWASP ModSecurity Core Rule Set Project Lead _______________________________________________ Owasp-modsecurity-core-rule-set mailing list [email protected] https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
