AW: Traffic monitoring
check out flowscan http://www.caida.org/tools/utilities/flowscan/ it gets close to what you want, assuming all the traffic is passing through a cisco router. Something like this for Linux would bei really cool ! Nik
Re: AW: Traffic monitoring
On Tue, 2003-03-18 at 16:04, debian-security wrote: check out flowscan http://www.caida.org/tools/utilities/flowscan/ it gets close to what you want, assuming all the traffic is passing through a cisco router. A better choice (IMHO) would be flow-tools at http://www.splintered.net/sw/flow-tools/ there is no debian package yet... but working on it :) Description: Flow-tools is library and a collection of programs used to collect, send, process, and generate reports from NetFlow data. The tools can be used together on a single server or distributed to multiple servers for large deployments. The flow-toools library provides an API for development of custom applications for NetFlow export versions 1,5,6 and the 14 currently defined version 8 subversions. A Perl and Python interface have been contributed and are included in the distribution. Flow data is collected and stored by default in host byte order, yet the files are portable across big and little endian architectures. Commands that utilize the network use a localip/remoteip/port designation for communication. localip is the IP address the host will use as a source for sending or bind to when receiving NetFlow PDU's (ie the destination address of the exporter. Configuring the localip to 0 will force the kernel to decide what IP address to use for sending and listen on all IP addresses for receiving. remoteip is the destination IP address used for sending or the expected address of the source when receiving. If the remoteip is 0 then the application will accept flows from any source address. The port is the UDP port number used for sending or receiving. When using multicast addresses the localip/remoteip/port is used to represent the source, group, and port respectively. -- JJ van Gorkum Knowledge Zone If UNIX isn't the solution, you've got the wrong problem.
Re: [d-security] Traffic monitoring
On Fri, Mar 14, 2003 at 08:03:17PM +0100, Nils wrote: How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. Try looking for netflow. It's a traffic accounting protocol invented by Cisco and supported by most Cisco routers and Linux via ntop or nprobe. Mostly used by ISPs. Accounts ip-proto,address,port,as-number,tcpflags,num_packets,num_bytes. It is normally installed on the border routers of the ISP facing the customer and the upstream providers interfaces (i.e. backbone traffic is not accounted) but you can alter this setup. bye, -christian- -- Christian Hammers WESTEND GmbH | Internet-Business-Provider Technik CISCO Systems Partner - Authorized Reseller Lütticher Straße 10 Tel 0241/701333-11 [EMAIL PROTECTED]D-52064 Aachen Fax 0241/911879
Re: Traffic monitoring
also sprach Adrian 'Dagurashibanipal' von Bidder [EMAIL PROTECTED] [2003.03.15.0135 +0100]: Hmmm. As long as you have specific protocols, you could always parse the server logs. ftp and http should be no problem, most daemons write a sensible log, I guess. Others (especially IMAP) I don't know. SSH probably doesn't write such a log. this will only be a rough estimate since it counts layer 7 traffic. for each TCP packet, you can add 44 bytes, and for each UDP packet another 28. if you need to do traffic accounting sensibly, there is no way around using separate IPs and/or an actual accounting device. -- Please do not CC me when replying to lists; I read them! .''`. martin f. krafft [EMAIL PROTECTED] : :' :proud Debian developer, admin, and user `. `'` `- Debian - when you have better things to do than fixing a system Keyserver problems? http://keyserver.kjsl.com/~jharris/keyserver.html Get my key here: http://people.debian.org/~madduck/gpg/330c4a75.asc pgp0.pgp Description: PGP signature
Re: Traffic monitoring
also sprach Adrian 'Dagurashibanipal' von Bidder [EMAIL PROTECTED] [2003.03.15.0135 +0100]: Hmmm. As long as you have specific protocols, you could always parse the server logs. ftp and http should be no problem, most daemons write a sensible log, I guess. Others (especially IMAP) I don't know. SSH probably doesn't write such a log. this will only be a rough estimate since it counts layer 7 traffic. for each TCP packet, you can add 44 bytes, and for each UDP packet another 28. if you need to do traffic accounting sensibly, there is no way around using separate IPs and/or an actual accounting device. -- Please do not CC me when replying to lists; I read them! .''`. martin f. krafft [EMAIL PROTECTED] : :' :proud Debian developer, admin, and user `. `'` `- Debian - when you have better things to do than fixing a system Keyserver problems? http://keyserver.kjsl.com/~jharris/keyserver.html Get my key here: http://people.debian.org/~madduck/gpg/330c4a75.asc pgpSSFLJ7FHUT.pgp Description: PGP signature
Re: Traffic monitoring
Am Saturday 15 March 2003 00:15 schrieb Stefan Neufeind: You might want to try out the packat iptraf and monitor the interface ipsec0. It gives you various overwiews on traffic going over each port in / out as well as other statistics. Only drawback: It only counts as long as you leave it running on console. [...] Hi, iptraf runs also in the background, try the -B option. Regards, Marcus -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
Am Saturday 15 March 2003 00:15 schrieb Stefan Neufeind: You might want to try out the packat iptraf and monitor the interface ipsec0. It gives you various overwiews on traffic going over each port in / out as well as other statistics. Only drawback: It only counts as long as you leave it running on console. [...] Hi, iptraf runs also in the background, try the -B option. Regards, Marcus
Traffic monitoring
Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... . . Workstation B... If I also could see what files being sent (names and sizes), it would be fantastic. Is it possible with SMB? (What about FTP, HTTP, RSYNC...) Of course I can't see what files get encapsulated in a SSH tunnel, but, I still want to know the volume and origin. Of course they can use different ports... This is not a police action I want to conduct, I just want a really strong position when complaints come from different directions. Those who pay say the cost is too high and those who use it say the connection is to slow. What the users don't realize is that if the costs isn't manageable, the ISP-connection will be cut off. They just blame each other for the volume sent/received. I'm just about feed up with it!!! As for now, all I have is a transparent squid and the total volume through the connection (with no separation on the volume the different workstations tribute). Can anyone at least solve some of my wishes? Forgive me my hard hidden frustration. Cheers - Nils Erikson -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
Nils wrote: Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). If you can install a machine as a sniffer (hubs only in the network, or a switch that supports port mirroring), iptraf may really help here. I don't find it very usefull over long trends, but I use iptraf on my network whenever I see an unexplained jump in traffic and need to track down the source. It's able to show traffic by port, by packet size, or a running display of source IP:port and destination IP:port pairs. Also supports packet filtering (which is really nice to filter out the port 22 connection from my workstation, so the continual screen updates don't distract me with increasing packet counts). It's also packaged for Debian. --Rich _ Rich Puhek ETN Systems Inc. 2125 1st Ave East Hibbing MN 55746 tel: 218.262.1130 email: [EMAIL PROTECTED] _ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
On 2003/03/14 08:03:17PM +0100, Fri, Nils wrote: How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). check out flowscan http://www.caida.org/tools/utilities/flowscan/ it gets close to what you want, assuming all the traffic is passing through a cisco router. it can get the type of traffic and the source/destination asn. andrew -- If there was any justice, my face would be on a bunch of crappy merchandise! --Homer Simpson Flaming Moe's pgp0.pgp Description: PGP signature
Re: Traffic monitoring
On Fri, Mar 14, 2003 at 08:03:17PM +0100, Nils wrote: We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Here is a quick and dirty method. I wanted to see what ICMP types where being used so I created a rule on my firewall for each type. The rule just returns, but the statistics (iptables -vnL) reveal the frequency of each type: # Not sure about these, start logging them... # find them in stats... iptables -N icmpwach for n in `seq 0 255`; do echo -n . iptables -I icmpwach -p icmp --icmp-type $n -j RETURN done ; echo iptables -I INPUT -j icmpwach iptables -I FORWARD -j icmpwach i=iptables -I INPUT -p icmp you might modify the loop to generate a return rule for each ip iptables -I bandwatch -s $n -p all -j RETURN iptables -I bandwatch -d $n -p all -j RETURN Then you could look at the iptable stats and see which ip is using the gateway. This might be more politically desirable than knowing the IP and the port ;) On the other hand you could come up with some ports and port ranges to monitor too. There are tons of software to calculate and make presentations of this kind of info. http://ipaudit.sourceforge.net/ipaudit-web/ Would you like to summarize and/or log network activity down to the ip address and port level of detail, but not record every packet? http://freshmeat.net/projects/traffacct/ www.hughes.com.au/products/traffacct/ TraffAcct is a network traffic accounting package designed to simplify the process of tracking and billing network usage. http://bubba.sourceforge.net/ Bandwidth Utilization Billing and Basic Accounting http://netacct-mysql.sourceforge.net/ bandwidth utilization, accounting Netacct-mySQL is a monitor which can log traffic generated by a specific network (incoming/outgoing). In fact it works like sniffer, puts network interface in PROMISC mode and collects traffic. http://torus.lnet.lut.fi/vnstat/ vnStat is a network traffic monitor for Linux that keeps a log of daily network traffic for the selected interface. http://ifmonitor.preteritoimperfeito.com/ ifmonitor is a simple network interface traffic logger and grapher for linux. gkrellm mrtg The list goes on, let us know what you come up with. // George -- GEORGE GEORGALIS, System Admin/Architectcell: 347-451-8229 Security Services, Web, Mail,mailto:[EMAIL PROTECTED] Multimedia, DB, DNS and Metrics. http://www.galis.org/george -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
## Nils ([EMAIL PROTECTED]): How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. If you are using kernel 2.4, you can use ulogd. If not, there is net-acct. net-acct might apear broken in debian stable, you may need the patch from http://exorsus.net/projects/net-acct/lockpatch.txt http://www.nadev.net/thomas/projects/nacctstats/ has a script for generating nice output. i am using net-acct, perl and PostgreSQL for monitoring about 200 hosts and about 50 gigabytes of traffic per day. The router is a Pentium-133 (32 MB RAM), the database runs on a PentiumIII-833 (512MB RAM, but there is a squid cache sitting on the same box). Every morning, the collected data gets copied to the database machine, where it is processed by a small (about 4kb, including report generation) perl script. The result are some tables showing network usage per host and per port (incoming and outgoing traffic seperated). My scripting is somewhat ugly, but perhaps it could be adapted with little effort. Scripts and some config available on request. There is currently no documentation as the whole thing was intended as dirty hack and not a full blown solution. Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... Generating such output would be a little more CPU intensive. Beware the amount of data you will generate. Expect several megabytes summary(!) per day. net-acct samples are about 30 megabytes a day in my setup. If I also could see what files being sent (names and sizes), it would be fantastic. Is it possible with SMB? (What about FTP, HTTP, RSYNC...) That requieres that the accounting tools know about all these protocols. Some sniffers are able to decode most protocols, but the selden do accounting. Of course I can't see what files get encapsulated in a SSH tunnel, but, I still want to know the volume and origin. Of course they can use different ports... This is not a police action I want to conduct, I just want a really strong position when complaints come from different directions. Those who pay say the cost is too high and those who use it say the connection is to slow. What the users don't realize is that if the costs isn't manageable, the ISP-connection will be cut off. They just blame each other for the volume sent/received. I'm just about feed up with it!!! Just show them the statistics. Or publish the daily Top 50... Be careful with the privacy of your users. Do not publish anything else than bytes per workstation. Perhaps it might be better to keep the statistics for yourself and talk to the biggest offenders directly. That depends on your environment. Regards, cmt -- Spare Space -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
On Fri, 14 Mar 2003 at 08:03:17PM +0100, Nils wrote: Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... . . IPTraf may do what you are looking for... -- Phil PGP/GPG Key: http://www.zionlth.org/~plhofmei/ wget -O - http://www.zionlth.org/~plhofmei/key.txt | gpg --import -- Excuse #24: radiosity depletion -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
You might want to try out the packat iptraf and monitor the interface ipsec0. It gives you various overwiews on traffic going over each port in / out as well as other statistics. Only drawback: It only counts as long as you leave it running on console. But I guess leaving it running for e.g. 12 hours (one work-day) should be sufficient to get an idea what's going on, right? And you could also try to sniff the SMB-traffic ... there are probably ways to listen which files (with what filenames etc.) are transfered. I strongly believe there are tools doing this out there. Ethereal maybe? (Haven't worked with it yet.) On 14 Mar 2003 at 20:03, Nils wrote: I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
While we're still in the field of counting and monitoring traffic: Is there any good way to account traffic on one computer by user? I searched several times for this but didn't find any good solution. Some people said it should be do-able with kernel-modules but nobody knew who had already done it. I have several users generating traffic over the network interface (eth0). What I would need is monitor incoming and outgoing traffic accounted by the uid the process is running to or from which the packets are received / sent. Hmm - did I at least make it a bit clear? Even if I have somebody running an ftp for getting or putting files ... or if I have someone using wget on the shell or getting remote-files via PHP or whatever I need to account this traffic to the uid - all on the local machine. And if I have someone opening a listening-port (this also appears with ftp-transfers) and waits for an incoming connection I would also like to bill the incoming connection to the same uid. That's my problem. Any good solutions out there? I'm stuck with this :-(( -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
On Sat, Mar 15, 2003 at 12:22:11AM +0100, Stefan Neufeind wrote: While we're still in the field of counting and monitoring traffic: Is there any good way to account traffic on one computer by user? I searched several times for this but didn't find any good solution. Some people said it should be do-able with kernel-modules but nobody knew who had already done it. I have several users generating traffic over the network interface (eth0). What I would need is monitor incoming and outgoing traffic accounted by the uid the process is running to or from which the packets are received / sent. Hmm - did I at least make it a bit clear? Even if I have somebody running an ftp for getting or putting files ... or if I have someone using wget on the shell or getting remote-files via PHP or whatever I need to account this traffic to the uid - all on the local machine. And if I have someone opening a listening-port (this also appears with ftp-transfers) and waits for an incoming connection I would also like to bill the incoming connection to the same uid. That's my problem. Any good solutions out there? I'm stuck with this :-(( Try ipac-ng: Description: IP Accounting for iptables( kernel =2.4) Can do accounting on any iptable rule (as I understand it). iptables have the capability to match on owner: iptables -A INPUT -m owner --uid-owner 2 Cheers Geoff Crompton -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
On Fri, Mar 14, 2003 at 10:39:59PM +0100, Christoph Moench-Tegeder wrote: If you are using kernel 2.4, you can use ulogd. I never got ulogd running properly. I'm running 0.97-1 from woody, and I was never able to get it to information to any files. Anyone want to comment on the following ulogd.conf file? nlgroup 1 logfile /var/log/ulogd.log loglevel 1 plugin /usr/lib/ulogd/ulogd_BASE.so syslogfile /var/log/ulogd.syslogemu syslogsync 1 plugin /usr/lib/ulogd/ulogd_LOGEMU.so dumpfile /var/log/ulogd.pktlog And I've got a filewall rule: -A INPUT -s 61.9.128.13 -i eth0 -p udp -m udp --dport 1024 -m limit --limit 20/hour -j ULOG --ulog-prefix BPA (Checking with iptables-save -c reveals that the rule has been getting matches). Geoff Crompton -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
On Fri, Mar 14, 2003 at 08:03:17PM +0100, Nils wrote: Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. try ntop traffic-vis darkstat regards Samuele -- When all the network has eyes, even if we were to send out minds turned into light or electrons... It is a time when one is not able to make a solid, a complex, into data yet... -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Traffic monitoring
On Sat, 2003-03-15 at 00:22, Stefan Neufeind wrote: Is there any good way to account traffic on one computer by user? I Hmmm. As long as you have specific protocols, you could always parse the server logs. ftp and http should be no problem, most daemons write a sensible log, I guess. Others (especially IMAP) I don't know. SSH probably doesn't write such a log. cheers -- vbi -- featured product: the GNU Compiler Collection - http://gcc.gnu.org signature.asc Description: This is a digitally signed message part
Traffic monitoring
Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... . . Workstation B... If I also could see what files being sent (names and sizes), it would be fantastic. Is it possible with SMB? (What about FTP, HTTP, RSYNC...) Of course I can't see what files get encapsulated in a SSH tunnel, but, I still want to know the volume and origin. Of course they can use different ports... This is not a police action I want to conduct, I just want a really strong position when complaints come from different directions. Those who pay say the cost is too high and those who use it say the connection is to slow. What the users don't realize is that if the costs isn't manageable, the ISP-connection will be cut off. They just blame each other for the volume sent/received. I'm just about feed up with it!!! As for now, all I have is a transparent squid and the total volume through the connection (with no separation on the volume the different workstations tribute). Can anyone at least solve some of my wishes? Forgive me my hard hidden frustration. Cheers - Nils Erikson
Re: Traffic monitoring
Nils wrote: Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). If you can install a machine as a sniffer (hubs only in the network, or a switch that supports port mirroring), iptraf may really help here. I don't find it very usefull over long trends, but I use iptraf on my network whenever I see an unexplained jump in traffic and need to track down the source. It's able to show traffic by port, by packet size, or a running display of source IP:port and destination IP:port pairs. Also supports packet filtering (which is really nice to filter out the port 22 connection from my workstation, so the continual screen updates don't distract me with increasing packet counts). It's also packaged for Debian. --Rich _ Rich Puhek ETN Systems Inc. 2125 1st Ave East Hibbing MN 55746 tel: 218.262.1130 email: [EMAIL PROTECTED] _
Re: Traffic monitoring
On 2003/03/14 08:03:17PM +0100, Fri, Nils wrote: How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). check out flowscan http://www.caida.org/tools/utilities/flowscan/ it gets close to what you want, assuming all the traffic is passing through a cisco router. it can get the type of traffic and the source/destination asn. andrew -- If there was any justice, my face would be on a bunch of crappy merchandise! --Homer Simpson Flaming Moe's pgpDcxiEC99ss.pgp Description: PGP signature
Re: Traffic monitoring
On Fri, Mar 14, 2003 at 08:03:17PM +0100, Nils wrote: We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Here is a quick and dirty method. I wanted to see what ICMP types where being used so I created a rule on my firewall for each type. The rule just returns, but the statistics (iptables -vnL) reveal the frequency of each type: # Not sure about these, start logging them... # find them in stats... iptables -N icmpwach for n in `seq 0 255`; do echo -n . iptables -I icmpwach -p icmp --icmp-type $n -j RETURN done ; echo iptables -I INPUT -j icmpwach iptables -I FORWARD -j icmpwach i=iptables -I INPUT -p icmp you might modify the loop to generate a return rule for each ip iptables -I bandwatch -s $n -p all -j RETURN iptables -I bandwatch -d $n -p all -j RETURN Then you could look at the iptable stats and see which ip is using the gateway. This might be more politically desirable than knowing the IP and the port ;) On the other hand you could come up with some ports and port ranges to monitor too. There are tons of software to calculate and make presentations of this kind of info. http://ipaudit.sourceforge.net/ipaudit-web/ Would you like to summarize and/or log network activity down to the ip address and port level of detail, but not record every packet? http://freshmeat.net/projects/traffacct/ www.hughes.com.au/products/traffacct/ TraffAcct is a network traffic accounting package designed to simplify the process of tracking and billing network usage. http://bubba.sourceforge.net/ Bandwidth Utilization Billing and Basic Accounting http://netacct-mysql.sourceforge.net/ bandwidth utilization, accounting Netacct-mySQL is a monitor which can log traffic generated by a specific network (incoming/outgoing). In fact it works like sniffer, puts network interface in PROMISC mode and collects traffic. http://torus.lnet.lut.fi/vnstat/ vnStat is a network traffic monitor for Linux that keeps a log of daily network traffic for the selected interface. http://ifmonitor.preteritoimperfeito.com/ ifmonitor is a simple network interface traffic logger and grapher for linux. gkrellm mrtg The list goes on, let us know what you come up with. // George -- GEORGE GEORGALIS, System Admin/Architectcell: 347-451-8229 Security Services, Web, Mail,mailto:[EMAIL PROTECTED] Multimedia, DB, DNS and Metrics. http://www.galis.org/george
Re: Traffic monitoring
## Nils ([EMAIL PROTECTED]): How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. If you are using kernel 2.4, you can use ulogd. If not, there is net-acct. net-acct might apear broken in debian stable, you may need the patch from http://exorsus.net/projects/net-acct/lockpatch.txt http://www.nadev.net/thomas/projects/nacctstats/ has a script for generating nice output. i am using net-acct, perl and PostgreSQL for monitoring about 200 hosts and about 50 gigabytes of traffic per day. The router is a Pentium-133 (32 MB RAM), the database runs on a PentiumIII-833 (512MB RAM, but there is a squid cache sitting on the same box). Every morning, the collected data gets copied to the database machine, where it is processed by a small (about 4kb, including report generation) perl script. The result are some tables showing network usage per host and per port (incoming and outgoing traffic seperated). My scripting is somewhat ugly, but perhaps it could be adapted with little effort. Scripts and some config available on request. There is currently no documentation as the whole thing was intended as dirty hack and not a full blown solution. Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... Generating such output would be a little more CPU intensive. Beware the amount of data you will generate. Expect several megabytes summary(!) per day. net-acct samples are about 30 megabytes a day in my setup. If I also could see what files being sent (names and sizes), it would be fantastic. Is it possible with SMB? (What about FTP, HTTP, RSYNC...) That requieres that the accounting tools know about all these protocols. Some sniffers are able to decode most protocols, but the selden do accounting. Of course I can't see what files get encapsulated in a SSH tunnel, but, I still want to know the volume and origin. Of course they can use different ports... This is not a police action I want to conduct, I just want a really strong position when complaints come from different directions. Those who pay say the cost is too high and those who use it say the connection is to slow. What the users don't realize is that if the costs isn't manageable, the ISP-connection will be cut off. They just blame each other for the volume sent/received. I'm just about feed up with it!!! Just show them the statistics. Or publish the daily Top 50... Be careful with the privacy of your users. Do not publish anything else than bytes per workstation. Perhaps it might be better to keep the statistics for yourself and talk to the biggest offenders directly. That depends on your environment. Regards, cmt -- Spare Space
Re: Traffic monitoring
On Fri, 14 Mar 2003 at 08:03:17PM +0100, Nils wrote: Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH... . . IPTraf may do what you are looking for... -- Phil PGP/GPG Key: http://www.zionlth.org/~plhofmei/ wget -O - http://www.zionlth.org/~plhofmei/key.txt | gpg --import -- Excuse #24: radiosity depletion
Re: Traffic monitoring
You might want to try out the packat iptraf and monitor the interface ipsec0. It gives you various overwiews on traffic going over each port in / out as well as other statistics. Only drawback: It only counts as long as you leave it running on console. But I guess leaving it running for e.g. 12 hours (one work-day) should be sufficient to get an idea what's going on, right? And you could also try to sniff the SMB-traffic ... there are probably ways to listen which files (with what filenames etc.) are transfered. I strongly believe there are tools doing this out there. Ethereal maybe? (Haven't worked with it yet.) On 14 Mar 2003 at 20:03, Nils wrote: I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. We have two computer labs, with its respective ISP-connections, both with volume based rates. These two sites are also connected to each other through a VPN. The volume between the two sites should really be marginal. Due to what we get charge by the ISP, we suspect a lot of non-sanctioned material (mp3..) being transported over smb. I would like to at least be able to monitor the volume from respective computer going through the firewall (and the VPN). Preferably, I would like to have information like: Date xx/xx/xx Workstation A (xxx.xxx.xxx.xxx) (95 MB) SMB.35 MB HTTP40 MB RSYNC...10 MB FTP..5 MB SSH...
Re: Traffic monitoring
While we're still in the field of counting and monitoring traffic: Is there any good way to account traffic on one computer by user? I searched several times for this but didn't find any good solution. Some people said it should be do-able with kernel-modules but nobody knew who had already done it. I have several users generating traffic over the network interface (eth0). What I would need is monitor incoming and outgoing traffic accounted by the uid the process is running to or from which the packets are received / sent. Hmm - did I at least make it a bit clear? Even if I have somebody running an ftp for getting or putting files ... or if I have someone using wget on the shell or getting remote-files via PHP or whatever I need to account this traffic to the uid - all on the local machine. And if I have someone opening a listening-port (this also appears with ftp-transfers) and waits for an incoming connection I would also like to bill the incoming connection to the same uid. That's my problem. Any good solutions out there? I'm stuck with this :-((
Re: Traffic monitoring
On Sat, Mar 15, 2003 at 12:22:11AM +0100, Stefan Neufeind wrote: While we're still in the field of counting and monitoring traffic: Is there any good way to account traffic on one computer by user? I searched several times for this but didn't find any good solution. Some people said it should be do-able with kernel-modules but nobody knew who had already done it. I have several users generating traffic over the network interface (eth0). What I would need is monitor incoming and outgoing traffic accounted by the uid the process is running to or from which the packets are received / sent. Hmm - did I at least make it a bit clear? Even if I have somebody running an ftp for getting or putting files ... or if I have someone using wget on the shell or getting remote-files via PHP or whatever I need to account this traffic to the uid - all on the local machine. And if I have someone opening a listening-port (this also appears with ftp-transfers) and waits for an incoming connection I would also like to bill the incoming connection to the same uid. That's my problem. Any good solutions out there? I'm stuck with this :-(( Try ipac-ng: Description: IP Accounting for iptables( kernel =2.4) Can do accounting on any iptable rule (as I understand it). iptables have the capability to match on owner: iptables -A INPUT -m owner --uid-owner 2 Cheers Geoff Crompton
Re: Traffic monitoring
On Fri, Mar 14, 2003 at 10:39:59PM +0100, Christoph Moench-Tegeder wrote: If you are using kernel 2.4, you can use ulogd. I never got ulogd running properly. I'm running 0.97-1 from woody, and I was never able to get it to information to any files. Anyone want to comment on the following ulogd.conf file? nlgroup 1 logfile /var/log/ulogd.log loglevel 1 plugin /usr/lib/ulogd/ulogd_BASE.so syslogfile /var/log/ulogd.syslogemu syslogsync 1 plugin /usr/lib/ulogd/ulogd_LOGEMU.so dumpfile /var/log/ulogd.pktlog And I've got a filewall rule: -A INPUT -s 61.9.128.13 -i eth0 -p udp -m udp --dport 1024 -m limit --limit 20/hour -j ULOG --ulog-prefix BPA (Checking with iptables-save -c reveals that the rule has been getting matches). Geoff Crompton
Re: Traffic monitoring
On Fri, Mar 14, 2003 at 08:03:17PM +0100, Nils wrote: Hello everybody! I have small but complicated problem. How do you monitor what network traffic you have and how much? I want to be able to see the origin and destination, type and volume. try ntop traffic-vis darkstat regards Samuele -- When all the network has eyes, even if we were to send out minds turned into light or electrons... It is a time when one is not able to make a solid, a complex, into data yet...
Re: Traffic monitoring
On Sat, 2003-03-15 at 00:22, Stefan Neufeind wrote: Is there any good way to account traffic on one computer by user? I Hmmm. As long as you have specific protocols, you could always parse the server logs. ftp and http should be no problem, most daemons write a sensible log, I guess. Others (especially IMAP) I don't know. SSH probably doesn't write such a log. cheers -- vbi -- featured product: the GNU Compiler Collection - http://gcc.gnu.org signature.asc Description: This is a digitally signed message part
Re: Network traffic monitoring. (which IP makes big traffic?)
On Thu, Dec 06, 2001 at 12:33:46AM -0800, Alvin Oga wrote: [snip] root# trafshow - shows in a small table ( more readable) the ongoing traffic ( keeps a ongoing total traffic Or try ntop . It has a web insterface and shows loads of various statistics. for the rest of the network monitoring tools.. http://www.Linux-Sec.net/Ethernet/ have fun alvin On Thu, 6 Dec 2001, Cho Yoonbae wrote: Hi, My network has been very slower than before. Someone suspected the virus like nimda. So I have to found out who makes very high traffic.. I am not network engineer now. What things I have to know? and Which softwares I can select? I'm asking advice for you. Have a nice day. byebye -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- GPG key-id: 1024D/DF04A255 Dmitriy AA16 8FAB 74E1 3511 83D0 9F4B F087 CEC9 DF04 A255 * encrypted personal mail is very much preferred * Free Dmitry Sklyarov! http://www.freesklyarov.org msg04679/pgp0.pgp Description: PGP signature
Re: Network traffic monitoring. (which IP makes big traffic?)
On Thu, Dec 06, 2001 at 12:33:46AM -0800, Alvin Oga wrote: [snip] root# trafshow - shows in a small table ( more readable) the ongoing traffic ( keeps a ongoing total traffic Or try ntop . It has a web insterface and shows loads of various statistics. for the rest of the network monitoring tools.. http://www.Linux-Sec.net/Ethernet/ have fun alvin On Thu, 6 Dec 2001, Cho Yoonbae wrote: Hi, My network has been very slower than before. Someone suspected the virus like nimda. So I have to found out who makes very high traffic.. I am not network engineer now. What things I have to know? and Which softwares I can select? I'm asking advice for you. Have a nice day. byebye -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED] -- GPG key-id: 1024D/DF04A255 Dmitriy AA16 8FAB 74E1 3511 83D0 9F4B F087 CEC9 DF04 A255 * encrypted personal mail is very much preferred * Free Dmitry Sklyarov! http://www.freesklyarov.org pgpqM4CFahyRb.pgp Description: PGP signature
Network traffic monitoring. (which IP makes big traffic?)
Hi, My network has been very slower than before. Someone suspected the virus like nimda. So I have to found out who makes very high traffic.. I am not network engineer now. What things I have to know? and Which softwares I can select? I'm asking advice for you. Have a nice day. byebye -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Network traffic monitoring. (which IP makes big traffic?)
hi cho easiest way is to run the simple tests first... root# tcpdump - watch for the ip# and between which 2 machines root# trafshow - shows in a small table ( more readable) the ongoing traffic ( keeps a ongoing total traffic for the rest of the network monitoring tools.. http://www.Linux-Sec.net/Ethernet/ have fun alvin On Thu, 6 Dec 2001, Cho Yoonbae wrote: Hi, My network has been very slower than before. Someone suspected the virus like nimda. So I have to found out who makes very high traffic.. I am not network engineer now. What things I have to know? and Which softwares I can select? I'm asking advice for you. Have a nice day. byebye -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Network traffic monitoring. (which IP makes big traffic?)
Hi, try iptraf, apt-get install iptraf -- Patrick Hsieh--[EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Network traffic monitoring. (which IP makes big traffic?)
-- Patrick Hsieh--[EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Network traffic monitoring. (which IP makes big traffic?)
Hi, My network has been very slower than before. Someone suspected the virus like nimda. So I have to found out who makes very high traffic.. I am not network engineer now. What things I have to know? and Which softwares I can select? I'm asking advice for you. Have a nice day. byebye
Re: Network traffic monitoring. (which IP makes big traffic?)
hi cho easiest way is to run the simple tests first... root# tcpdump - watch for the ip# and between which 2 machines root# trafshow - shows in a small table ( more readable) the ongoing traffic ( keeps a ongoing total traffic for the rest of the network monitoring tools.. http://www.Linux-Sec.net/Ethernet/ have fun alvin On Thu, 6 Dec 2001, Cho Yoonbae wrote: Hi, My network has been very slower than before. Someone suspected the virus like nimda. So I have to found out who makes very high traffic.. I am not network engineer now. What things I have to know? and Which softwares I can select? I'm asking advice for you. Have a nice day. byebye
Re: Network traffic monitoring. (which IP makes big traffic?)
Hi, try iptraf, apt-get install iptraf -- Patrick Hsieh--[EMAIL PROTECTED]
Re: Network traffic monitoring. (which IP makes big traffic?)
-- Patrick Hsieh--[EMAIL PROTECTED]
