Alaerte,
> Is it wise
to export flows from 25 routers to a Linux machine
> with Intel 4Ghz, 1GMbytes RAM and 200Gbytes of HD?
> with Intel 4Ghz, 1GMbytes RAM and 200Gbytes of HD?
> The
routers are 7609 with high traffic.
Short
answer: Yes
Long
answer:
The number of
unique routers sending your server Netflow is not really all that
important. Neither is total bytes routed. One NT domain controller's
200byte chit chat to all and sundry dwarf the number of flows you'll see on
backup VLANs. flow-capture's performance is dependant on the peak number
of flows received per minute regardless of how many router's are sending flows
or how large the value of "doctets" in the flow records. In practice you will
never see flow-capture's peak performance as your additional management and
analysis of the flow data will require much more cpu and
memory.
For example, my
2x1.2GHz AIX box with 1GB of RAM captures up to 750Kflows in each 300 sec file.
(2500 flows/s). The CPU usage of flow-capture is always less than
1%. The 100Mbit/s NIC runs at 1.0 to 1.6Mbit/s which is pretty much what
is written to disk (11GB per day!). I estimate that I could get at least 10
times this without any collection
issues.
The performance
issue is running the cron jobs to extract the interesting data from the
flows. (Dodgy source IP address, big WAN users, top senders of un-ACKed
SYN flows, the stats the the latest whiz-bang application etc). I have 300
seconds to run about 30 scripts against a 75MB flow file before compressing it
and sending it to the archive.
A few
tips:
1) Make
flow-capture runs in uncompressed mode. Only compress the file after you're done
with it. (Maybe a week or so after you captured the data).
2) Keen an eye on the output of "netstat -s" to ensure you're not losing flows. You may want to tweak your inbound buffer from your ethernet card to deal with the bursts of UDP MSFCs like to deliver. For example:
echo 1048576 > /proc/sys/net/core/rmem_max
echo 1048576 > /proc/sys/net/core/wmem_max
echo 50 > /proc/sys/net/unix/max_dgram_qlen
3) Target your flow feeds to a dedicated NIC. Send your FTP extracts and backups to a separate card.
2) Keen an eye on the output of "netstat -s" to ensure you're not losing flows. You may want to tweak your inbound buffer from your ethernet card to deal with the bursts of UDP MSFCs like to deliver. For example:
echo 1048576 > /proc/sys/net/core/rmem_max
echo 1048576 > /proc/sys/net/core/wmem_max
echo 50 > /proc/sys/net/unix/max_dgram_qlen
3) Target your flow feeds to a dedicated NIC. Send your FTP extracts and backups to a separate card.
4) Graph your CPU, NIC and Disk usage with MRTG so you
can see when your box is going to break before it
does.
5)
Ensure the flow-capture process is running at a higher priority than flow-cat
etc.
Cheers
Alistair
**********************************************************************
Registered Office:
Marks and Spencer plc
Waterside House
35 North Wharf Road
London
W2 1NW
Registered No. 214436 in England and Wales.
Telephone (020) 7935 4422
Facsimile (020) 7487 2670
<<www.marksandspencer.com>>
Please note that electronic mail may be monitored.
This e-mail is confidential. If you received it by mistake, please let us know and then delete it from your system; you should not copy, disclose, or distribute its contents to anyone nor act in reliance on this e-mail, as this is prohibited and may be unlawful.
2005
_______________________________________________ Flow-tools mailing list [EMAIL PROTECTED] http://mailman.splintered.net/mailman/listinfo/flow-tools
