Thank you for your reply.
I searched the archive, and I found your answer:
*That's what you can define with the waterwark. Assume you have a size limit
of 100MB and a watermark of 90%. Expiring data starts when the size is over
the limit of 100MB. It will delete data down to 90% of 100MB = 90MB.
So the data size will oscillate between 90MB and 100MB. If you have lifetime
limits the same applies here: 30days are 720h. 90% is 648h. So your data
oscillates between 648h and 720h.*
*The oscillation is mostly given by the level of the watermark you define in
nfsen.conf.*
So, from what you're saying, the used space would never go past the profile
size. Since I have only one profile (live), the only logical explanation
would be that I got ~20G of data for the whole profile in 5 minutes. This
caused the disk to overflow and the cleaning mechanism didn't manage to do
its job.
I find it very hard to believe that I get a spike in the number of flows (of
20G) in just 5 minutes. I've looked over the traffic graphs for the
collector's interface and even if there is an increase in traffic compared
to week days, there is no such spike that would clarify the situation...
I'll have to lower the maximum space for the profile, but this means it's
just unused space... :(
Regards,
Adrian
On Tue, Apr 22, 2008 at 11:31 AM, Peter Haag <[EMAIL PROTECTED]> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Adrian,
>
> - --On April 21, 2008 13:33:09 +0300 Adrian Popa <[EMAIL PROTECTED]>
> wrote:
>
> | Hello everybody,
> |
> | I ran into the same problem I had a while ago - nfsen sometimes (during
> the
> | weekend) runs out of disk space and is at a stand-still...
> |
> | Here's how my system looked this morning:
> |
> | [EMAIL PROTECTED] ~]# df -h
> | Filesystem Size Used Avail Use% Mounted on
> | /dev/sda1 15G 8.4G 5.4G 61% /
> | */dev/sda2 241G 229G 0 100% /data*
> | none 3.5G 0 3.5G 0% /dev/shm
> | */dev/sda5 13G 13G 0 100% /var*
> |
> | The /var partition was filled by the syslog error messages:
> | Apr 20 11:19:49 localhost /usr/local/bin/nfcapd[4490]: Process_v9:
> output
> | buffer size error. Abort v9 record processing
> | Apr 20 11:19:49 localhost /usr/local/bin/nfcapd[4487]: Failed to write
> | output buffer to disk: 'No space left on device'
> |
> | I have a script that checks for partitions overflowing, but for some
> reason
> | it didn't do the trick this time...
> |
> | After deleting some old flow data and clearing the /var partition, I
> rebuilt
> | the live profile and restarted nfsen...
> |
> | [EMAIL PROTECTED] clearOldRRDs]# ./emergencyCleanup.pl
> | /data 100
> | /var 100
> | Deleting files older than 2 days
> | /data 48
> | /var 85
> | [EMAIL PROTECTED] clearOldRRDs]# df -h
> | Filesystem Size Used Avail Use% Mounted on
> | /dev/sda1 15G 8.4G 5.4G 61% /
> | */dev/sda2 241G 110G 120G 48% /data*
> | none 3.5G 0 3.5G 0% /dev/shm
> | */dev/sda5 13G 11G 1.9G 85% /var*
> | [EMAIL PROTECTED] bin]# ./nfsen -r live
> | name live
> | group (nogroup)
> | tcreate Thu Apr 10 15:02:27 2008
> | tstart Thu Apr 17 23:40:00 2008
> | tend Mon Apr 21 09:25:00 2008
> | updated Mon Apr 21 09:25:00 2008
> | expire 3 days 0 hours
> | size 100.4 GB
> | *maxsize 200.0 GB*
> | type live
> | locked 0
> | status OK
> | version 130
> |
> |
> | My question is: if the maximum size of the profile is 200G, and the full
> | profile would occupy 210G, why did it fill my 230G partition?
>
> The channel size is updated from each collector as the files are stored on
> disk. The accumulated profile size is calculated by
> nfexpire at each 5min cycle run. If you manipulate anything by hand
> change/add/delete files, then you need to rebuild to total
> profile size, as NfSen has no ideas about any manual changes. Please also
> note, that NfSen handles profiles sizes on each
> individual bases and not according to the available size on a volume.
>
> |
> | Could you explain agian how the watermarking works? I would like to set
> it
> | to delete stuff as soon as possible and not be lazy...
>
> I posted an explanation a month ago to this list, don't know the exact
> date. It should be in the archive.
>
> - Peter
> |
> | Thank you,
> | Adrian
>
>
>
> - --
> _______ SWITCH - The Swiss Education and Research Network ______
> Peter Haag, Security Engineer, Member of SWITCH CERT
> PGP fingerprint: D9 31 D5 83 03 95 68 BA FB 84 CA 94 AB FC 5D D7
> SWITCH, Werdstrasse 2, P.O. Box, CH-8021 Zurich, Switzerland
> E-mail: [EMAIL PROTECTED] Web: http://www.switch.ch/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.3 (Darwin)
>
> iQCVAwUBSA2idv5AbZRALNr/AQKm7wP/cLP7M7cpeRCnRf3xIk8zZSRP6SBS40Wm
> aNala997gCka+XiuoUm7qD9asmE2vGqPiyXTJGxQSJn6Pzw5Qrts7IlmVXJ9UdVG
> 03/LXWdkH6xdBkDXoOXg1KsuBoSxZo4XbJ/eVEghOGhc9Qf8Qh9hYLVBEoMNxRhs
> SAZG5NLsV74=
> =ZGSr
> -----END PGP SIGNATURE-----
>
>
-------------------------------------------------------------------------
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
_______________________________________________
Nfsen-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
