Re: [ossec-list] Very big syscheck queue - how to deal with it?
On Thu, Apr 20, 2017 at 2:07 PM, B. S. wrote: >> To clear a syscheck db: >> 1. stop the ossec processes on the server >> 2. /var/ossec/bin/syscheck_control -u AGENT_ID >> 3. Start the ossec processes on the server > > Thank you - "To clear a syscheck db" gave me the context needed to better > understand syscheck_control --help. > > So: >> 2. /var/ossec/bin/syscheck_control -u AGENT_ID > > Could also have been '/var/ossec/bin/syscheck_control -u all'. > > Granted, you could not know I'm on a local install with no remote agents. > > However, sadly, /var/ossec/queue is still quite large. Per the OP. > > In particular, in my case, /var/ossec/queue/diff > And there is the context that could have made this much easier. > What is the appropriate way to squish this dir down? > (Corresponding question would then be, specifying a 'checkpoint'? Is that > even possible - to, say, say maintain your original, or, say, as of 1 month > ago, so that diffs between then and now are kept?) > > I see references to just deleting files in diff. Is that safe? i.e. Won't > befuddle ossec? (I get I'll lose the change history.) > You should be able to delete the files. I don't generally use the diff option, so haven't tested this all myself. > > > On 04/20/2017 01:29 PM, dan (ddp) wrote: >> >> On Thu, Apr 20, 2017 at 1:02 PM, Bee esS wrote: If you need them shrunk, you'll have to clear the databases. >>> >>> >>> How? >>> >> >> When resurrecting 2+ year old threads, it might be best to offer more >> context. >> To clear a syscheck db: >> 1. stop the ossec processes on the server >> 2. /var/ossec/bin/syscheck_control -u AGENT_ID >> 3. Start the ossec processes on the server >> >> >>> On Monday, 8 December 2014 08:03:57 UTC-5, dan (ddpbsd) wrote: On Mon, Dec 8, 2014 at 7:17 AM, horst knete wrote: > > > Now looking at our /var/ossec/queue/syscheck queue directory at the > server, > this folder has an size of 5.4 GB and contains 2 "files" for almost > every > ossec-client. Those files grow because there are changes on the systems they represent. Those are the syscheck database files. If you need them shrunk, you'll have to clear the databases. I don't know what the rest of that means though (the cpu/ram/network stuff). > Because none of these lacks are given i´d like to know how i can > decrease > the size of this queue. >>> -- >>> >>> --- >>> You received this message because you are subscribed to the Google Groups >>> "ossec-list" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to ossec-list+unsubscr...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >> >> > > -- > > --- You received this message because you are subscribed to the Google > Groups "ossec-list" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to ossec-list+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ossec-list] Re: Deleting the OSSEC agent 'queue' directory
Answers inline below.
On 4/20/17, 2:17 PM, "ossec-list@googlegroups.com on behalf of B. S."
wrote:
Kewl! Thank you!
> Here is my solution if you are using active response and allow remote
commands.
Ah, but reading it, you also answer local installs! Thank you!
So, just deleting files in /var/ossec/queue/diff/local/ won't befuddle
ossec? – I had the same concern but have not ran into any major issues with
deleting this directory. I have run into a couple of minor issues where a file
changes but you do not receive the diff in the alerts log. I believe this is
because once you delete the ‘diff/local’ directory, the agent does not have the
previous version of the file to perform the diff.
What are the consequences / impact? Loss of change history, presumably. –
As long as all of the events/logs are being forwarded to the OSSEC server you
should not have any issues. This directory only keeps the diffs and the current
state of the files that are being monitored with the ‘report_changes’ option.
Once a change has been found the diff is forwarded to the OSSEC console.
er, more precisely, loss of change history between versions at a point
in time, I guess.
On 04/20/2017 01:41 PM, Patrick Tobin wrote:
> Here is my solution if you are using active response and allow remote
commands.
>
> AR Script (/var/ossec/active-response/bin/fix-var.sh)
>
> #!/bin/bash
> ARCommand='rm -rf /var/ossec/queue/diff/local/'
> RDate=`date`
> LOG=/var/ossec/logs/ar.log
> date >> ${LOG}
> $ARCommand >> ${LOG}
>
> AR Rule (/var/ossec/rules/local_rules.xml)
>
>
> 530
> ossec: output: 'df /var':
> DiskFull
> /var is getting full. Clearing logs
> low_diskspace,
>
>
> Command (/var/ossec/etc/shared/agent.conf)
>
>
> command
> Disk=`df /var | tail -1 | awk '{print $2}'`;vSize=`du
/var/ossec/queue/diff/local | awk '{print $1}' | tail -1`;dStatus=`df -h /var |
tail -1`;dStatus=`df -h /var | tail -1`;if [[ ${percent} -gt "75" ]]; then echo
-n "DiskFull ${dStatus}";fi
> 360
> df /var
>
>
> Note: You can change the percentage at which this is activated to fit
your environment. (if [[ ${percent} -gt "75" ]]) Change the ‘75’ to the
percentage you would like it to activate.
>
>
>
> From: on behalf of Bee esS
> Reply-To: "ossec-list@googlegroups.com"
> Date: Thursday, April 20, 2017 at 1:03 PM
> To: ossec-list
> Subject: [ossec-list] Re: Deleting the OSSEC agent 'queue' directory
>
> Bump.
>
> On Wednesday, 19 August 2015 10:51:26 UTC-4, Jamey B wrote:
> I'm making a CRON job to remove anything in the queue folder, would this
be a good CRON job if I wanted the directory cleared if the items are over 5
days old and I want it ran once a day at 10PM? The last time I took my OSSEC
server down, the agent disk space started getting too big in
/var/ossec/queue/diff/local after a few weeks. Would any other directories do
the same thing, or is this the only directory that gets queue data?
>
> 0 22 * * * /usr/bin/find /var/ossec/queue/diff/local/* -mtime +5 -exec rm
{} \;
>
>
> I don't want the OSSEC agent to take up a lot of disk space, what else
could I do?
> --
>
> ---
> You received this message because you are subscribed to the Google Groups
"ossec-list" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to
ossec-list+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ossec-list] Re: Deleting the OSSEC agent 'queue' directory
Kewl! Thank you!
> Here is my solution if you are using active response and allow remote
commands.
Ah, but reading it, you also answer local installs! Thank you!
So, just deleting files in /var/ossec/queue/diff/local/ won't befuddle
ossec?
What are the consequences / impact? Loss of change history, presumably.
er, more precisely, loss of change history between versions at a point
in time, I guess.
On 04/20/2017 01:41 PM, Patrick Tobin wrote:
Here is my solution if you are using active response and allow remote commands.
AR Script (/var/ossec/active-response/bin/fix-var.sh)
#!/bin/bash
ARCommand='rm -rf /var/ossec/queue/diff/local/'
RDate=`date`
LOG=/var/ossec/logs/ar.log
date >> ${LOG}
$ARCommand >> ${LOG}
AR Rule (/var/ossec/rules/local_rules.xml)
530
ossec: output: 'df /var':
DiskFull
/var is getting full. Clearing logs
low_diskspace,
Command (/var/ossec/etc/shared/agent.conf)
command
Disk=`df /var | tail -1 | awk '{print $2}'`;vSize=`du /var/ossec/queue/diff/local | awk
'{print $1}' | tail -1`;dStatus=`df -h /var | tail -1`;dStatus=`df -h /var | tail -1`;if [[ ${percent} -gt
"75" ]]; then echo -n "DiskFull ${dStatus}";fi
360
df /var
Note: You can change the percentage at which this is activated to fit your environment.
(if [[ ${percent} -gt "75" ]]) Change the ‘75’ to the percentage you would like
it to activate.
From: on behalf of Bee esS
Reply-To: "ossec-list@googlegroups.com"
Date: Thursday, April 20, 2017 at 1:03 PM
To: ossec-list
Subject: [ossec-list] Re: Deleting the OSSEC agent 'queue' directory
Bump.
On Wednesday, 19 August 2015 10:51:26 UTC-4, Jamey B wrote:
I'm making a CRON job to remove anything in the queue folder, would this be a
good CRON job if I wanted the directory cleared if the items are over 5 days
old and I want it ran once a day at 10PM? The last time I took my OSSEC server
down, the agent disk space started getting too big in
/var/ossec/queue/diff/local after a few weeks. Would any other directories do
the same thing, or is this the only directory that gets queue data?
0 22 * * * /usr/bin/find /var/ossec/queue/diff/local/* -mtime +5 -exec rm {} \;
I don't want the OSSEC agent to take up a lot of disk space, what else could
I do?
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups "ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ossec-list] Very big syscheck queue - how to deal with it?
> To clear a syscheck db: > 1. stop the ossec processes on the server > 2. /var/ossec/bin/syscheck_control -u AGENT_ID > 3. Start the ossec processes on the server Thank you - "To clear a syscheck db" gave me the context needed to better understand syscheck_control --help. So: > 2. /var/ossec/bin/syscheck_control -u AGENT_ID Could also have been '/var/ossec/bin/syscheck_control -u all'. Granted, you could not know I'm on a local install with no remote agents. However, sadly, /var/ossec/queue is still quite large. Per the OP. In particular, in my case, /var/ossec/queue/diff What is the appropriate way to squish this dir down? (Corresponding question would then be, specifying a 'checkpoint'? Is that even possible - to, say, say maintain your original, or, say, as of 1 month ago, so that diffs between then and now are kept?) I see references to just deleting files in diff. Is that safe? i.e. Won't befuddle ossec? (I get I'll lose the change history.) On 04/20/2017 01:29 PM, dan (ddp) wrote: On Thu, Apr 20, 2017 at 1:02 PM, Bee esS wrote: If you need them shrunk, you'll have to clear the databases. How? When resurrecting 2+ year old threads, it might be best to offer more context. To clear a syscheck db: 1. stop the ossec processes on the server 2. /var/ossec/bin/syscheck_control -u AGENT_ID 3. Start the ossec processes on the server On Monday, 8 December 2014 08:03:57 UTC-5, dan (ddpbsd) wrote: On Mon, Dec 8, 2014 at 7:17 AM, horst knete wrote: Now looking at our /var/ossec/queue/syscheck queue directory at the server, this folder has an size of 5.4 GB and contains 2 "files" for almost every ossec-client. Those files grow because there are changes on the systems they represent. Those are the syscheck database files. If you need them shrunk, you'll have to clear the databases. I don't know what the rest of that means though (the cpu/ram/network stuff). Because none of these lacks are given i´d like to know how i can decrease the size of this queue. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [ossec-list] Re: Deleting the OSSEC agent 'queue' directory
Here is my solution if you are using active response and allow remote commands.
AR Script (/var/ossec/active-response/bin/fix-var.sh)
#!/bin/bash
ARCommand='rm -rf /var/ossec/queue/diff/local/'
RDate=`date`
LOG=/var/ossec/logs/ar.log
date >> ${LOG}
$ARCommand >> ${LOG}
AR Rule (/var/ossec/rules/local_rules.xml)
530
ossec: output: 'df /var':
DiskFull
/var is getting full. Clearing logs
low_diskspace,
Command (/var/ossec/etc/shared/agent.conf)
command
Disk=`df /var | tail -1 | awk '{print $2}'`;vSize=`du
/var/ossec/queue/diff/local | awk '{print $1}' | tail -1`;dStatus=`df -h /var |
tail -1`;dStatus=`df -h /var | tail -1`;if [[ ${percent} -gt "75" ]]; then echo
-n "DiskFull ${dStatus}";fi
360
df /var
Note: You can change the percentage at which this is activated to fit your
environment. (if [[ ${percent} -gt "75" ]]) Change the ‘75’ to the percentage
you would like it to activate.
From: on behalf of Bee esS
Reply-To: "ossec-list@googlegroups.com"
Date: Thursday, April 20, 2017 at 1:03 PM
To: ossec-list
Subject: [ossec-list] Re: Deleting the OSSEC agent 'queue' directory
Bump.
On Wednesday, 19 August 2015 10:51:26 UTC-4, Jamey B wrote:
I'm making a CRON job to remove anything in the queue folder, would this be a
good CRON job if I wanted the directory cleared if the items are over 5 days
old and I want it ran once a day at 10PM? The last time I took my OSSEC server
down, the agent disk space started getting too big in
/var/ossec/queue/diff/local after a few weeks. Would any other directories do
the same thing, or is this the only directory that gets queue data?
0 22 * * * /usr/bin/find /var/ossec/queue/diff/local/* -mtime +5 -exec rm {} \;
I don't want the OSSEC agent to take up a lot of disk space, what else could I
do?
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ossec-list] Very big syscheck queue - how to deal with it?
On Thu, Apr 20, 2017 at 1:02 PM, Bee esS wrote: >> If you need them shrunk, you'll have to clear the databases. > > How? > When resurrecting 2+ year old threads, it might be best to offer more context. To clear a syscheck db: 1. stop the ossec processes on the server 2. /var/ossec/bin/syscheck_control -u AGENT_ID 3. Start the ossec processes on the server > On Monday, 8 December 2014 08:03:57 UTC-5, dan (ddpbsd) wrote: >> >> On Mon, Dec 8, 2014 at 7:17 AM, horst knete wrote: >> > >> > Now looking at our /var/ossec/queue/syscheck queue directory at the >> > server, >> > this folder has an size of 5.4 GB and contains 2 "files" for almost >> > every >> > ossec-client. >> >> Those files grow because there are changes on the systems they >> represent. Those are the syscheck database files. If you need them >> shrunk, you'll have to clear the databases. I don't know what the rest >> of that means though (the cpu/ram/network stuff). >> >> > Because none of these lacks are given i´d like to know how i can >> > decrease >> > the size of this queue. >> > -- > > --- > You received this message because you are subscribed to the Google Groups > "ossec-list" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to ossec-list+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ossec-list] Re: Deleting the OSSEC agent 'queue' directory
Bump.
On Wednesday, 19 August 2015 10:51:26 UTC-4, Jamey B wrote:
>
> I'm making a CRON job to remove anything in the queue folder, would this
> be a good CRON job if I wanted the directory cleared if the items are over
> 5 days old and I want it ran once a day at 10PM? The last time I took my
> OSSEC server down, the agent disk space started getting too big in
> /var/ossec/queue/diff/local after a few weeks. Would any other directories
> do the same thing, or is this the only directory that gets queue data?
>
> 0 22 * * * /usr/bin/find /var/ossec/queue/diff/local/* -mtime +5 -exec rm
> {} \;
>
>
> I don't want the OSSEC agent to take up a lot of disk space, what else
> could I do?
>
--
---
You received this message because you are subscribed to the Google Groups
"ossec-list" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to ossec-list+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [ossec-list] Very big syscheck queue - how to deal with it?
> If you need them shrunk, you'll have to clear the databases. How? On Monday, 8 December 2014 08:03:57 UTC-5, dan (ddpbsd) wrote: > > On Mon, Dec 8, 2014 at 7:17 AM, horst knete > wrote: > > > > Now looking at our /var/ossec/queue/syscheck queue directory at the > server, > > this folder has an size of 5.4 GB and contains 2 "files" for almost > every > > ossec-client. > > Those files grow because there are changes on the systems they > represent. Those are the syscheck database files. If you need them > shrunk, you'll have to clear the databases. I don't know what the rest > of that means though (the cpu/ram/network stuff). > > > Because none of these lacks are given i´d like to know how i can > decrease > > the size of this queue. > > -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ossec-list] Override eventlog with eventchannel via Centralized agent config
I wasn't sure how to do this or if it's possible but I have a large number of ossec agents where I want to filter out specific Windows Event ID agent side. If I modify the ossec.conf on the agent and replace the log_format of my System from eventlog to eventchannel it works however if I leave it to eventlog and alter the centralized agent config to include that for Windows OS it doesn't work. I do see it get replicated to the agent under the shared folder but it looks like eventlog is taking priority. Touching each agent is not feasible as I just don't have that kind of control, at least I would have to somehow repackage an ossec install and wrap a new config into it, then have my IT people reinstall it on hundreds of Windows systems. Although I'm testing filtering event ID 7000 on a workstation I have many Windows servers with the windows packet filtering bombarding the event logs. This ends up saturating my network links from the agent to the manager which I want to eliminate. In ossec.conf System eventlog In Shared folder as agent.conf System eventchannel Event/System[EventID!=7000] -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ossec-list] Alerts.log loop
Hi everyone, My OS - Ubuntu Server 16.04 LTS, OSSEC version - 2.9.0 Today I encountered strange error with OSSEC - log monitoring ended up in a loop constantly checking portion of logs between 6AM and 9AM, sending duplicate emails of course. Troubleshooting the issue I noticed: 1) Logs monitored seem to be ok, nothing exceptional in them, 2) Analysisd was using 100% cpu on a core it is bound to, 3) Restarting ossec solved a problem. What could be the reason? Could it be Syscheckd kicking in every 6 hours? I've noticed this in ossec.log: 2017/04/20 06:52:48 ossec-syscheckd: INFO: Starting syscheck scan (forwarding database). 2017/04/20 06:52:48 ossec-syscheckd: INFO: Starting syscheck database (pre-scan). But syscheck is checking only limited set of files on two webserver's directories. It's not very much. -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[ossec-list] Re: Rule 510 is triggering events but logtest is not showing any rules that should be triggered
Hi Rob, I guess your decoder is working in ossec-logtest. But, in the real world, the events from rootcheck don't use xml decoders, they use c decoders: https://github.com/wazuh/wazuh/blob/master/src/analysisd/analysisd.c#L772 Regards. On Tuesday, April 18, 2017 at 7:45:13 PM UTC+2, Rob Williams wrote: > > Hi Jesus, > > I'll try this and let you know. The decoder is extracting the ID > effectively, however, it is just not matching the log with any rule it > seems. > > On Tuesday, April 18, 2017 at 5:14:28 AM UTC-7, Jesus Linares wrote: >> >> Hi Rob, >> >> you need to add the conditions to trigger that rule only for your >> specific files. Use match or regex: >> >> >> 510 >> >> Ignore rule 510 for 600 seconds for some files. >> >> >> >> I think you can't use *same_id *because the decoders are not extracting >> any ID. >> >> Regards. >> >> On Monday, April 17, 2017 at 6:55:19 PM UTC+2, Rob Williams wrote: >>> >>> Hi Jesus, the first rule is what I am trying. You said I can match the >>> file in but can I do that when the file changes as is not one file >>> I want to ignore. Can I use regex syntax in rules? I used it in decoders as >>> I thought I wasn't able to. Thanks! >>> >>> >>> 510 >>> >>> Ignore rule 510 for 600 seconds if the same ID is >>> matched. >>> >>> >>> On Monday, April 17, 2017 at 3:16:48 AM UTC-5, Jesus Linares wrote: What rule did you use?. Please, share here the rule and the alerts that you want to ignore. I'd need the ID from the decoder to do so There are no xml decoders for rootcheck. What you want to extract in the id field is the file, right?. You can do a *match* in the rule for the file. Regards. On Friday, April 14, 2017 at 12:13:50 AM UTC+2, Rob Williams wrote: > > Hi Jesus, > > Thanks for the reply. I have noticed when I activate this rule, it > blocks all events and does not alert on the first event. Also note, I am > trying to use the ID field from my decoder to match against. I can't just > use a static match as the ID continuously changes so I'd need the ID from > the decoder to do so. Any ideas? Thanks! > > On Wednesday, April 5, 2017 at 12:26:31 PM UTC-7, Rob Williams wrote: >> >> Hi all, >> >> I'm running into an issue where rule 510 is triggering and I'm >> getting spammed with alerts but I can't seem to tune it correctly. >> What's >> weird is that I am still getting alerted for rule 510 for this log, but >> I >> can't figure out how to get that to show in logtest. Basically, I am >> getting spammed with rule 510 and trying to filter it down more and here >> is >> what happens when I enter the log in logtest: any ideas on how >> to >> fix this? >> >> **Phase 1: Completed pre-decoding. >> >>full event: 'File '/filepath/' is owned by root and has >> written permissions to anyone.' >> >>hostname: 'hostname' >> >>program_name: '(null)' >> >>log: 'File '/filepath/' is owned by root and has written >> permissions to anyone.' >> >> >> **Phase 2: Completed decoding. >> >>decoder: 'sample_decoder_setup' >> >>id: '/filepath/' >> > -- --- You received this message because you are subscribed to the Google Groups "ossec-list" group. To unsubscribe from this group and stop receiving emails from it, send an email to ossec-list+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
