Thanks a lot Dan : it works ! I just changed : <regex>"\w+ (\S+) HTTP\S+ (\d+) |</regex> in : <regex>"\w+ (\.+) HTTP\S+ (\d+) |</regex> to get the url with spaces inside and I built a rule to catch the url with "epub" or "mobi" to get an alert when the Calibre Web Server serves contents for people outside our lan.
Well, may I ask another question : - These logs lines come from a log file : "server_access_log.txt" that I monitor with a <location> directive in "ossec.conf" - There is no "distinctive" part in these logs such as a progname, so it is very difficult to identify logs lines coming from the calibre-content-server - It looks like a classic "web-access" log format without the timezone part - Is it possible to have a decoder specific to a single file ? for exemple : <localfile> <log_format>syslog</log_format> <decoder>specific.decoder.xml</decoder> <location>/home/calibre/.config/calibre/server_access_log.txt</location> </localfile> Thanks again, Alain 2011/8/13 dan (ddp) <ddp...@gmail.com> > Sorry about that. Try modifying web-accesslog to look like the > following instead (remove the calibre one): > <decoder name="web-accesslog"> > <type>web-log</type> > <prematch>^\d+.\d+.\d+.\d+ |^::ffff:\d+.\d+.\d+.\d+ </prematch> > <regex>^(\d+.\d+.\d+.\d+) \S+ \S+ [\S+ \S\d+] </regex> > <regex>"\w+ (\S+) HTTP\S+ (\d+) |</regex> > <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "\S+ (\S+) > HTTP\S+" (\d+) </regex> > <order>srcip, url, id</order> > </decoder> > > > > On Sat, Aug 13, 2011 at 3:55 AM, Alain SPAITE <spaite.al...@fremenil.com> > wrote: > > Thanks dan for your quick answer. > > I did add your calibre-decoder to local_rules.xml > > > > <decoder name="calibre"> > > <parent>web-accesslog</parent> > > <type>web-log</type> > > <prematch>^\S+ \S+ \S+ </prematch> > > <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "(\S+) (\S+) > > HTTP\S+" (\d+) </regex> > > <order>srcip,action,url,id</order> > > </decoder> > > > > Here is what I got when I test a NCSA formatted log : > > > > 83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET /get/epub/331 > HTTP/1.1" > > 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" > > > > **Phase 1: Completed pre-decoding. > > full event: '83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET > > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ > > Reader(Android)"' > > hostname: 'ns1' > > program_name: '(null)' > > log: '83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET > > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ > > Reader(Android)"' > > > > **Phase 2: Completed decoding. > > decoder: 'web-accesslog' > > > > **Rule debugging: > > Trying rule: 4 - Generic template for all web rules. > > *Rule 4 matched. > > *Trying child rules. > > Trying rule: 31100 - Access log messages grouped. > > *Rule 31100 matched. > > *Trying child rules. > > Trying rule: 31108 - Ignored URLs (simple queries). > > Trying rule: 31115 - URL too long. Higher than allowed on most > browsers. > > Possible attack. > > Trying rule: 31103 - SQL injection attempt. > > Trying rule: 31104 - Common web attack. > > Trying rule: 31105 - XSS (Cross Site Scripting) attempt. > > Trying rule: 31101 - Web server 400 error code. > > Trying rule: 31120 - Web server 500 error code (server error). > > > > **Phase 3: Completed filtering (rules). > > Rule id: '31100' > > Level: '0' > > Description: 'Access log messages grouped.' > > > > It looks like srcip,id and url are no more decoded on these "classical" > > logs. > > > > Here is what I got when I test a "Calibre" formatted log : > > > > 83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331 HTTP/1.1" > 200 > > 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" > > > > > > **Phase 1: Completed pre-decoding. > > full event: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET > > /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ > > Reader(Android)"' > > hostname: 'ns1' > > program_name: '(null)' > > log: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331 > > HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"' > > > > **Phase 2: Completed decoding. > > decoder: 'web-accesslog' > > srcip: '83.233.145.196' > > action: 'GET' > > url: '/get/epub/331' > > id: '200' > > > > **Rule debugging: > > Trying rule: 4 - Generic template for all web rules. > > *Rule 4 matched. > > *Trying child rules. > > Trying rule: 31100 - Access log messages grouped. > > *Rule 31100 matched. > > *Trying child rules. > > Trying rule: 31108 - Ignored URLs (simple queries). > > *Rule 31108 matched. > > > > **Phase 3: Completed filtering (rules). > > Rule id: '31108' > > Level: '0' > > Description: 'Ignored URLs (simple queries).' > > > > The "calibre" decoder did not fire. But, the srcip, action, url and id > are > > extracted from the "calibre" log. > > I would like : > > - the NCSA logs decoded like before with the "web-accesslog" decoder with > > all the relevant info > > - the "calibre" logs decoded with the "calibre" decoder so I could write > a > > rule with <decoded_as>calibre</decoded_as> > > > > Thanks a lot for your help. > > > > > > 2011/8/12 dan (ddp) <ddp...@gmail.com> > >> > >> <decoder name="calibre"> > >> <parent>web-accesslog</parent> > >> <type>web-log</type> > >> <prematch>^\S+ \S+ \S+ </prematch> > >> <regex>^(\S+) \S+ \S+ [\d\d/\S+/\d\d\d\d:\d\d:\d\d:\d\d] "(\S+) > >> (\S+) HTTP\S+" (\d+) </regex> > >> <order>srcip,action,url,id</order> > >> </decoder> > >> > >> > >> On Fri, Aug 12, 2011 at 4:21 PM, Alain SPAITE <alain.spa...@gmail.com> > >> wrote: > >> > Hi everyone, > >> > > >> > I'm new to Ossec configuration and I try to check the logs for a > Calibre > >> > content server (http://calibre-ebook.com/). > >> > This content server works on the CherryPy web server written in > Python. > >> > The log format does not include the timezone info : > >> > > >> >> 83.233.145.196 - - [10/Jul/2011:22:57:31] "GET /get/epub/331 > HTTP/1.1" > >> >> 200 > >> >> 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" > >> > > >> > instead of the NCSA common log format which would be : > >> > > >> >> 83.233.145.196 - - [10/Jul/2011:22:57:31 +0200] "GET /get/epub/331 > >> >> HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)" > >> > > >> > Here is what I got with this format with the classic "web-accesslog" > >> > decoder > >> > : > >> > > >> >> **Phase 1: Completed pre-decoding. > >> >> full event: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET > >> >> /get/epub/331 HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ > >> >> Reader(Android)"' > >> >> hostname: 'ns1' > >> >> program_name: '(null)' > >> >> log: '83.233.145.196 - - [10/Jul/2011:22:57:31] "GET > >> >> /get/epub/331 > >> >> HTTP/1.1" 200 9607 "" "Stanza iPhone/Aldiko/Moon+ Reader(Android)"' > >> >> > >> >> **Phase 2: Completed decoding. > >> >> decoder: 'web-accesslog' > >> >> > >> >> **Rule debugging: > >> >> Trying rule: 4 - Generic template for all web rules. > >> >> *Rule 4 matched. > >> >> *Trying child rules. > >> >> Trying rule: 31100 - Access log messages grouped. > >> >> *Rule 31100 matched. > >> >> *Trying child rules. > >> >> Trying rule: 31108 - Ignored URLs (simple queries). > >> >> Trying rule: 31115 - URL too long. Higher than allowed on most > >> >> browsers. Possible attack. > >> >> Trying rule: 31103 - SQL injection attempt. > >> >> Trying rule: 31104 - Common web attack. > >> >> Trying rule: 31105 - XSS (Cross Site Scripting) attempt. > >> >> Trying rule: 31101 - Web server 400 error code. > >> >> Trying rule: 31120 - Web server 500 error code (server error). > >> >> > >> >> **Phase 3: Completed filtering (rules). > >> >> Rule id: '31100' > >> >> Level: '0' > >> >> Description: 'Access log messages grouped.' > >> >> > >> > > >> > The souceip,url and id are no extracted. > >> > Then, I tried to write a new decoder for the Calibre-CherryPy format : > >> > > >> >> <decoder name="calibre-accesslog"> > >> >> <type>web-log</type> > >> >> <parent>web-accesslog</parent> > >> >> <prematch>^\d+.\d+.\d+.\d+ |^::ffff:\d+.\d+.\d+.\d+ </prematch> > >> >> <regex>^(\d+.\d+.\d+.\d+) \S+ \S+ [\S+] </regex> > >> >> <regex>"\w+ (\S+) HTTP\S+ (\d+) </regex> > >> >> <order>srcip, url, id</order> > >> >> </decoder> > >> > > >> > But it doesn't work and whichever log format I sent (NCSA or CherryPy) > >> > it is > >> > always the "web-accesslog" that fired... > >> > Any help would be welcome ! > >> > You could find more logs samples and results on this pastebin : > >> > http://pastebin.archlinux.fr/433501 > >> > > >> > Thanks, > >> > Alain > >> > > >> > > > > > >