Good Morning All, I sit writing this while I have a "large" query (for us ;) running on our NetFlow data. Here at Lancaster we record NF from both our border and our ResNet border (for the purpose of recording NAT translations).
Currently running nfdump-snapshot-20070312 on one of our flows servers which is a Dell 2950 with 2 x 2.4Ghz Xeons (w/ Dual Cores), 4 x 500Gb (1 System, 3 Data in FreeBSD gvinum striped). When performing a simple query, I see the three data disks at about 11Mb/s each, and one of the CPU cores at 7%. The result of said query is: Total flows processed: 345346192, skipped: 0, Bytes read: 17958283900 Sys: 67.092s flows/second: 5147290.2 Wall: 827.917s flows/second: 417126.3 Running a few rudimentary tests (which I understand are misrepresentive). # dd if=/dev/zero of=test bs=128k count=30000 3932160000 bytes transferred in 38.009780 secs (103451270 bytes/sec) systat -vmstat shows all three data disks writing data at about 35Mb/s each. # dd of=/dev/null if=test bs=128k count=30000 3932160000 bytes transferred in 25.982248 secs (151340253 bytes/sec) systat -vmstat shows all three data disks reading at about 40Mb/s each. Obviously nfdump is reading from smaller files (in this case about 2.5Mb on average). Does anyone have any suggestions for optimisation of either FreeBSD or nfSen? Peter, I've had a quick look at nfdump's source code, I think that you're reading a record in at a time in small chunks, thus making many calls to fread/read making many calls per block. I can't guarantee any time due to other work commitments, but maybe looking at reading in larger blocks or maybe something like mmap might help? Now that being said more recent versions might be different. Of course as always thanks for a great application, nfSen has made such a different to how and when we use flows, it really is excellent. Any information would be appreciated. Kind regards, Peter. -- Peter A. Wood e: [EMAIL PROTECTED] Network Security Specialist t: +44 1524 510153 Technical Services Group Lancaster University PGP Fingerprint: C3B5 376B 16C5 F8D1 E3AE 617F 5718 C338 1D06 0689
signature.asc
Description: OpenPGP digital signature
------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
_______________________________________________ Nfsen-discuss mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nfsen-discuss
