On Sun, Aug 14, 2011 at 9:13 AM, Alain SPAITE <spaite.al...@fremenil.com> wrote: > Thanks a lot Dan : it works ! > I just changed : > <regex>"\w+ (\S+) HTTP\S+ (\d+) |</regex> > in : > <regex>"\w+ (\.+) HTTP\S+ (\d+) |</regex> > to get the url with spaces inside and I built a rule to catch the url with > "epub" or "mobi" to get an alert when the Calibre Web Server serves contents > for people outside our lan. >
I figured spaces would get changed to %20 or whatever. Interesting. > Well, may I ask another question : > - These logs lines come from a log file : "server_access_log.txt" that I > monitor with a <location> directive in "ossec.conf" > - There is no "distinctive" part in these logs such as a progname, so it is > very difficult to identify logs lines coming from the calibre-content-server > - It looks like a classic "web-access" log format without the timezone part > - Is it possible to have a decoder specific to a single file ? for exemple : > <localfile> > <log_format>syslog</log_format> > <decoder>specific.decoder.xml</decoder> > <location>/home/calibre/.config/calibre/server_access_log.txt</location> > </localfile> > Nope, no way to do this. > Thanks again, > Alain > > 2011/8/13 dan (ddp) <ddp...@gmail.com> >> >> Sorry about that. Try modifying web-accesslog to look like the >> following instead (remove the calibre one): >> <decoder name="web-accesslog"> >> <type>web-log</type> >> <prematch>^\d+.\d+.\d+.\d+ |^::ffff:\d+.\d+.\d+.\d+ </prematch> >> <regex>^(\d+.\d+.\d+.\d+) \S+ \S+ [\S+ \S\d+] </regex> >> <regex>"\w+ (\S+) HTTP\S+ (\d+) |</regex> >> <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "\S+ (\S+) >> HTTP\S+" (\d+) </regex> >> <order>srcip, url, id</order> >> </decoder> >> >> >> >> On Sat, Aug 13, 2011 at 3:55 AM, Alain SPAITE <spaite.al...@fremenil.com> >> wrote: >> > Thanks dan for your quick answer. >> > I did add your calibre-decoder to local_rules.xml >> > >> > <decoder name="calibre"> >> > <parent>web-accesslog</parent> >> > <type>web-log</type> >> > <prematch>^\S+ \S+ \S+ </prematch> >> > <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "(\S+) (\S+) >> > HTTP\S+" (\d+) </regex> >> > <order>srcip,action,url,id</order> >> > </decoder> >> > >> > Here is what I got when I test a NCSA formatted log : >> > >> > 83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET /get/epub/331 >> > HTTP/1.1" >> > 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" >> > >> > **Phase 1: Completed pre-decoding. >> > full event: '83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET >> > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ >> > Reader(Android)"' >> > hostname: 'ns1' >> > program_name: '(null)' >> > log: '83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET >> > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ >> > Reader(Android)"' >> > >> > **Phase 2: Completed decoding. >> > decoder: 'web-accesslog' >> > >> > **Rule debugging: >> > Trying rule: 4 - Generic template for all web rules. >> > *Rule 4 matched. >> > *Trying child rules. >> > Trying rule: 31100 - Access log messages grouped. >> > *Rule 31100 matched. >> > *Trying child rules. >> > Trying rule: 31108 - Ignored URLs (simple queries). >> > Trying rule: 31115 - URL too long. Higher than allowed on most >> > browsers. >> > Possible attack. >> > Trying rule: 31103 - SQL injection attempt. >> > Trying rule: 31104 - Common web attack. >> > Trying rule: 31105 - XSS (Cross Site Scripting) attempt. >> > Trying rule: 31101 - Web server 400 error code. >> > Trying rule: 31120 - Web server 500 error code (server error). >> > >> > **Phase 3: Completed filtering (rules). >> > Rule id: '31100' >> > Level: '0' >> > Description: 'Access log messages grouped.' >> > >> > It looks like srcip,id and url are no more decoded on these "classical" >> > logs. >> > >> > Here is what I got when I test a "Calibre" formatted log : >> > >> > 83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331 HTTP/1.1" >> > 200 >> > 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" >> > >> > >> > **Phase 1: Completed pre-decoding. >> > full event: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET >> > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ >> > Reader(Android)"' >> > hostname: 'ns1' >> > program_name: '(null)' >> > log: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET >> > /get/epub/331 >> > HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"' >> > >> > **Phase 2: Completed decoding. >> > decoder: 'web-accesslog' >> > srcip: '83.233.145.196' >> > action: 'GET' >> > url: '/get/epub/331' >> > id: '200' >> > >> > **Rule debugging: >> > Trying rule: 4 - Generic template for all web rules. >> > *Rule 4 matched. >> > *Trying child rules. >> > Trying rule: 31100 - Access log messages grouped. >> > *Rule 31100 matched. >> > *Trying child rules. >> > Trying rule: 31108 - Ignored URLs (simple queries). >> > *Rule 31108 matched. >> > >> > **Phase 3: Completed filtering (rules). >> > Rule id: '31108' >> > Level: '0' >> > Description: 'Ignored URLs (simple queries).' >> > >> > The "calibre" decoder did not fire. But, the srcip, action, url and id >> > are >> > extracted from the "calibre" log. >> > I would like : >> > - the NCSA logs decoded like before with the "web-accesslog" decoder >> > with >> > all the relevant info >> > - the "calibre" logs decoded with the "calibre" decoder so I could write >> > a >> > rule with <decoded_as>calibre</decoded_as> >> > >> > Thanks a lot for your help. >> > >> > >> > 2011/8/12 dan (ddp) <ddp...@gmail.com> >> >> >> >> <decoder name="calibre"> >> >> <parent>web-accesslog</parent> >> >> <type>web-log</type> >> >> <prematch>^\S+ \S+ \S+ </prematch> >> >> <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "(\S+) >> >> (\S+) HTTP\S+" (\d+) </regex> >> >> <order>srcip,action,url,id</order> >> >> </decoder> >> >> >> >> >> >> On Fri, Aug 12, 2011 at 4:21 PM, Alain SPAITE <alain.spa...@gmail.com> >> >> wrote: >> >> > Hi everyone, >> >> > >> >> > I'm new to Ossec configuration and I try to check the logs for a >> >> > Calibre >> >> > content server (http://calibre-ebook.com/). >> >> > This content server works on the CherryPy web server written in >> >> > Python. >> >> > The log format does not include the timezone info : >> >> > >> >> >> 83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331 >> >> >> HTTP/1.1" >> >> >> 200 >> >> >> 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" >> >> > >> >> > instead of the NCSA common log format which would be : >> >> > >> >> >> 83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET /get/epub/331 >> >> >> HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" >> >> > >> >> > Here is what I got with this format with the classic "web-accesslog" >> >> > decoder >> >> > : >> >> > >> >> >> **Phase 1: Completed pre-decoding. >> >> >> full event: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET >> >> >> /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ >> >> >> Reader(Android)"' >> >> >> hostname: 'ns1' >> >> >> program_name: '(null)' >> >> >> log: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET >> >> >> /get/epub/331 >> >> >> HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"' >> >> >> >> >> >> **Phase 2: Completed decoding. >> >> >> decoder: 'web-accesslog' >> >> >> >> >> >> **Rule debugging: >> >> >> Trying rule: 4 - Generic template for all web rules. >> >> >> *Rule 4 matched. >> >> >> *Trying child rules. >> >> >> Trying rule: 31100 - Access log messages grouped. >> >> >> *Rule 31100 matched. >> >> >> *Trying child rules. >> >> >> Trying rule: 31108 - Ignored URLs (simple queries). >> >> >> Trying rule: 31115 - URL too long. Higher than allowed on most >> >> >> browsers. Possible attack. >> >> >> Trying rule: 31103 - SQL injection attempt. >> >> >> Trying rule: 31104 - Common web attack. >> >> >> Trying rule: 31105 - XSS (Cross Site Scripting) attempt. >> >> >> Trying rule: 31101 - Web server 400 error code. >> >> >> Trying rule: 31120 - Web server 500 error code (server error). >> >> >> >> >> >> **Phase 3: Completed filtering (rules). >> >> >> Rule id: '31100' >> >> >> Level: '0' >> >> >> Description: 'Access log messages grouped.' >> >> >> >> >> > >> >> > The souceip,url and id are no extracted. >> >> > Then, I tried to write a new decoder for the Calibre-CherryPy format >> >> > : >> >> > >> >> >> <decoder name="calibre-accesslog"> >> >> >> <type>web-log</type> >> >> >> <parent>web-accesslog</parent> >> >> >> <prematch>^\d+.\d+.\d+.\d+ |^::ffff:\d+.\d+.\d+.\d+ </prematch> >> >> >> <regex>^(\d+.\d+.\d+.\d+) \S+ \S+ [\S+] </regex> >> >> >> <regex>"\w+ (\S+) HTTP\S+ (\d+) </regex> >> >> >> <order>srcip, url, id</order> >> >> >> </decoder> >> >> > >> >> > But it doesn't work and whichever log format I sent (NCSA or >> >> > CherryPy) >> >> > it is >> >> > always the "web-accesslog" that fired... >> >> > Any help would be welcome ! >> >> > You could find more logs samples and results on this pastebin : >> >> > http://pastebin.archlinux.fr/433501 >> >> > >> >> > Thanks, >> >> > Alain >> >> > >> >> > >> > >> > > >
