Re: [pmacct-discussion] multiple interfaces & uni-directional flows
Hi Stig, Very briefly to confirm: a) you are correct, libpcap captures both inbound and outbound traffic and b) the workaround you have put in place not only makes sense but is also by far the most efficient way to filter traffic out of pmacctd. Cheers, Paolo On Tue, Aug 04, 2009 at 10:39:00AM -0700, Stig Thormodsrud wrote: > I notice with multiple interfaces that I get duplicate flows. If I recall > correctly a cisco router does netflow only on input while it seems pcap > captures both inbound & outbound packets. My work around to filter out > the output flows was to use a pcap_filter such as: > > ! > daemonize: true > promisc: false > pidfile: /var/run/pmacctd-eth0.pid > imt_path: /tmp/pmacctd-eth0.pipe > plugins: nfprobe, memory > aggregate: src_host,dst_host,src_port,dst_port,proto,tos,flows,tag > interface: eth0 > syslog: daemon > ! filter out packets with the mac address of eth0 > pcap_filter: !ether src 00:0c:29:8c:53:7c > nfprobe_receiver: 172.16.117.25:2100 > nfprobe_version: 5 > nfprobe_engine: 1:2 > post_tag: 2 > > > Is this the approach others are using with multiple interfaces or is there > a better way? > > Thanks, > > stig ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] multiple interfaces & uni-directional flows
I notice with multiple interfaces that I get duplicate flows. If I recall correctly a cisco router does netflow only on input while it seems pcap captures both inbound & outbound packets. My work around to filter out the output flows was to use a pcap_filter such as: ! daemonize: true promisc: false pidfile: /var/run/pmacctd-eth0.pid imt_path: /tmp/pmacctd-eth0.pipe plugins: nfprobe, memory aggregate: src_host,dst_host,src_port,dst_port,proto,tos,flows,tag interface: eth0 syslog: daemon ! filter out packets with the mac address of eth0 pcap_filter: !ether src 00:0c:29:8c:53:7c nfprobe_receiver: 172.16.117.25:2100 nfprobe_version: 5 nfprobe_engine: 1:2 post_tag: 2 Is this the approach others are using with multiple interfaces or is there a better way? Thanks, stig ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Hi Mariano, maybe irrelevant for your scenario but pmacct version 0.11.5 is able to make the most profit by the fields of the DLT_LINUX_SLL header - which is prepended to the L3 header when using the "any" interface. This is an extract from the ChangeLog: + pmacctd, the Linux Cooked device (DLT_LINUX_SLL) handler has been enhanced by supporting 'src_mac' and 'vlan' aggregation primitives. Cheers, Paolo On Fri, Jan 23, 2009 at 01:36:40PM +0100, Mariano Spadaccini wrote: > Chris Wilson wrote: > > > Have you tried using "any" as the interface name to capture all flows? > > Yes, but... > --- > r...@mixer# pmacctd -i any -c > src_mac,dst_mac,src_host,dst_host,src_port,dst_port > WARN ( cmdline ): No plugin has been activated; defaulting to in-memory > table. > OK ( default/memory ): waiting for data on: '/tmp/collect.pipe' > OK ( default/core ): link type is: 113 > ERROR ( default/core ): MAC aggregation not available for link type: 113 > --- > > src_mac and dst_mac are important data in my log. > > However this machine is nat-firewall (loc <-> fw <-> net) > My interest is only for traffic through interfaces loc <-> fw, > before nat process. > > Cheers, > Mariano ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Chris Wilson wrote:
> Have you tried using "any" as the interface name to capture all flows?
Yes, but...
---
r...@mixer# pmacctd -i any -c
src_mac,dst_mac,src_host,dst_host,src_port,dst_port
WARN ( cmdline ): No plugin has been activated; defaulting to in-memory
table.
OK ( default/memory ): waiting for data on: '/tmp/collect.pipe'
OK ( default/core ): link type is: 113
ERROR ( default/core ): MAC aggregation not available for link type: 113
---
src_mac and dst_mac are important data in my log.
However this machine is nat-firewall (loc <-> fw <-> net)
My interest is only for traffic through interfaces loc <-> fw,
before nat process.
Cheers,
Mariano
---
($nn='tux300181')=~s/(.)/$b{$a{$1}++}--;$1/eg;@b=map{-$_}values(%b);
print $nn,'.'x3,' ',$b[1],' <> chars, ',$b[0]+$b[1]," total chars\t";
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Hi Mariano, On Fri, 23 Jan 2009, Mariano Spadaccini wrote: > Now the problem is only on the tagged port. But I have tried others > probe, with the same error (only unidirectional flows). > > However I have resolved with one pmacctd/one interface (untagged port). Have you tried using "any" as the interface name to capture all flows? I think it should work, although it will not put any interface into promiscuous mode. Please let us know if it does work. Cheers, Chris. -- Aptivate | http://www.aptivate.org | Phone: +44 1223 760887 The Humanitarian Centre, Fenner's, Gresham Road, Cambridge CB1 2ES Aptivate is a not-for-profit company registered in England and Wales with company number 04980791. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Paolo Lucente wrote: > two possible issues pop up to my mind: > > a) pmacct configuration issue: the memory plugin is queried by the >client through a pipe file. Such pipe file has to be different for >each instance of the plugin - otherwise the last activated takes >control of the default pipe file. Try adding to your configuration >a line like this: > >imt_path: /tmp/acct_eth0.77.pipe > >then you can query the specific instance from the client with the >following commandline: > >pmacct -p /tmp/acct_eth0.77.pipe Yeah.. this was the problem! > > b) network issue: what happens if you bind tcpdump to the interface >instead? Do you see any differences with what is reported by pmacct >(maybe running in parallel)? > No problem here... tcpdump is correct. Now the problem is only on the tagged port. But I have tried others probe, with the same error (only unidirectional flows). However I have resolved with one pmacctd/one interface (untagged port). Thank you for your rapid reply... Cheers, Mariano --- perluniintro - Unicode is a character set standard which plans to codify all of the writing systems of the world, plus many other symbols. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Hi Mariano, two possible issues pop up to my mind: a) pmacct configuration issue: the memory plugin is queried by the client through a pipe file. Such pipe file has to be different for each instance of the plugin - otherwise the last activated takes control of the default pipe file. Try adding to your configuration a line like this: imt_path: /tmp/acct_eth0.77.pipe then you can query the specific instance from the client with the following commandline: pmacct -p /tmp/acct_eth0.77.pipe b) network issue: what happens if you bind tcpdump to the interface instead? Do you see any differences with what is reported by pmacct (maybe running in parallel)? Let me know. Cheers, Paolo On Thu, Jan 22, 2009 at 05:39:50PM +0100, Mariano Spadaccini wrote: > Hi, > > I am a new user of pmacct and I use Ubuntu 8.04LTS. > I have searched some info in archive, but I haven't find any answer. > > I have tried to capture flow on vlan tagged port eth0 (801.1q), but all > flows are unidirectional, from internet/router to lan. > So I have tried to put N pmacctd daemons on my untagged ports eth0.n (n > = 1..N) but I have captured only one interface eth0.x, where x is N > (last n). > In this case, the flow is complete (internet to lan and lan to > internet), but it is capured only one interface. > > This is my configuration of untagged interface eth0.77 > (/etc/pmacct/pmacctd.eth0.77.conf) > -- > daemonize: true > pidfile: /var/run/pmacctd.eth0.77.pid > aggregate: src_mac,dst_mac,src_host,dst_host,src_port,dst_port > interface: eth0.77 > plugins: memory > -- > > All interface are configurated like this conf... any idea? > > Thank you for your response, > Mariano ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] multiple interfaces
Hi,
I am a new user of pmacct and I use Ubuntu 8.04LTS.
I have searched some info in archive, but I haven't find any answer.
I have tried to capture flow on vlan tagged port eth0 (801.1q), but all
flows are unidirectional, from internet/router to lan.
So I have tried to put N pmacctd daemons on my untagged ports eth0.n (n
= 1..N) but I have captured only one interface eth0.x, where x is N
(last n).
In this case, the flow is complete (internet to lan and lan to
internet), but it is capured only one interface.
This is my configuration of untagged interface eth0.77
(/etc/pmacct/pmacctd.eth0.77.conf)
--
daemonize: true
pidfile: /var/run/pmacctd.eth0.77.pid
aggregate: src_mac,dst_mac,src_host,dst_host,src_port,dst_port
interface: eth0.77
plugins: memory
--
All interface are configurated like this conf... any idea?
Thank you for your response,
Mariano
---
s;;13456?13689=14400$d=2601-2304+2304z2401|3136,2401$TeX$LaTeX;;s;(\d+)*?;push
@_,$1;eg;map{print chr(sqrt($_)),' 'x1} @_;
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Hi Anil and Juan, On Wed, 7 Jan 2009, Juan Rivera wrote: > My understanding is that any one instance of the daemon can only bind to > a single interface. I think that a workaround would be to run more than > one instance of the daemon, one per interface, and use a different > configuration file for each instance. tcpdump can bind to all interfaces but it can't put them all into promiscuous mode at the same time. If that's OK for your application, try using the device "any" instead of a real device. Cheers, Chris. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] multiple interfaces
Hello, My understanding is that any one instance of the daemon can only bind to a single interface. I think that a workaround would be to run more than one instance of the daemon, one per interface, and use a different configuration file for each instance. Juan Anil wrote: > I am running pmacct as a daemon. > How can I tell it to monitor multiple interfaces? > > Simply specifying: > interfaces: e1000g0,e1000g3 > > didn't work. > > Thanks, > Anil > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists > ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] multiple interfaces
I am running pmacct as a daemon. How can I tell it to monitor multiple interfaces? Simply specifying: interfaces: e1000g0,e1000g3 didn't work. Thanks, Anil ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
Hi Jim,
> I have been testing pmacct quite a bit and, using the primitives as you
> and the docs describe, I have been able to separate the in/out traffic and
> filter it. I can do this with the print plugin and the postgresql plugin.
>
> The netflow and sflow plugins work slightly differently from each other,
> but neither quite right.
>
> When I use my Cisco 7200 series router to generate netflow data and point
> it at my software (in this case, Netflow Analyzer from Manage Engine) the
> software accepts the stream, logs the data and and draws a single graph
> for each interface. The single graph has both the "in" traffic and the
> "out" traffic graphed all on a single graph.
I think you must set appropriate parameters on your routers as we do for
'nfprobe' plugin:
KEY: nfprobe_engine
DESC: allows to define Engine ID and Engine Type fields. It applies only to
NetFlow v5 and v9. In v9, the supplied value fills last two bytes of
SourceID
field. Expects two non-negative numbers, up to 255 each and separated by the
":" symbol.
!!! It also allows a collector to distinguish between distinct probe
instances running on the same box; this is also important for letting
NetFlow v9 templates to work correctly: in fact, template IDs get
automatically
selected only inside single daemon instances. (default: 0:0) !!!
> So here is my problem. Let's say I have eth0 on my PC based router.
> Setting up pmacct as you describe and using the netflow plugin does result
> in a netlow stream I can direct to my collector software. However, the
> same software now draws a graph with all the traffic from that interface
> as "in" and shows no traffic as "out." The in appears to be a combination
> of the in and out traffic.
Sorry, i don't understand about what software ("collector", "same") you
tell. Is this not 'pmacct'?
> The sflow plugin works slightly differently. Using sflow, the software
> will draw one graph for "in" and another graph for "out" treating the two
> as separate interfaces.
>
> If I try to monitor several interfaces, say eth0, eth1 and eth2, the
> netflow plugin seems to produce a flow that only reports a single
> interface, while the sflow plugin reports all three interfaces to the
> software. Still, I can't get in and out data on the same graph.
>
> It seems I am not the only person having this problem. There was a little
> discussion of it on the Netflow Analyzer forum:
>
> http://forums.adventnet.com/viewtopic.php?t=355981
>
>>From looking at the data, pmacct clearly knows what traffic is in and what
> is out. From the reading I have done it seems that the software that
> collects the data wants separate flows for each direction that refer to
> the same interface.
>
> So, is it possible to make pmacct generate separate flows for in and out
> for the same interface?
Yes, of course, i am using it so now. If you set 'memory' plugin and
start
'pmacctd' daemon in debug mode:
debug: true
daemonize: false
plugins: memory[in], memory[out]
you can see as it create different flows and collect data to them
separately
for in and out traffic (if you have relative settings).
Alex
--
Кредит на развитие бизнеса! Индивидуальным предпринимателям и юр. лицам.
Специальные предложения: 'Кредит на приобретение коммерческого автомобиля',
'Кредит руководителю'. Белросбанк, (017)287-66-97, http://www.belrosbank.by
___
pmacct-discussion mailing list
http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
In 'CONFIG-KEYS' described follow method: "sum_ are compound primitives which allow to join inbound and outbound traffic into a single aggregate." Alex >>When you have both in and out traffic separately you can also use >> 'net' aggragation for accumulate them in common statistics (see docs, i >> don't test that variant): >> >> aggregate[all-1]: net, ... > >Sorry, i am wrong. Of course, we can't use 'src_net' and 'dst_net' for > accumulate input and output traffic. >But we can make appropriate SQL 'select' request for join both type of > traffic in one digit. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
>When you have both in and out traffic separately you can also use > 'net' aggragation for accumulate them in common statistics (see docs, i > don't test that variant): > > aggregate[all-1]: net, ... Sorry, i am wrong. Of course, we can't use 'src_net' and 'dst_net' for accumulate input and output traffic. But we can make appropriate SQL 'select' request for join both type of traffic in one digit. Alex ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
Hi Jim, If you want to gather both in and out traffic you must use follow instructions: plugins: nfprobe aggregate_filter: net 72.46.0.0/17 or more complex plugins: nfprobe[in], nfprobe[out], nfprobe[all] aggregate_filter[in]: dst net 72.46.0.0/17 aggregate_filter[out]: src net 72.46.0.0/17 aggregate_filter[all]: net 72.46.0.0/17 When you have both in and out traffic separately you can also use 'net' aggragation for accumulate them in common statistics (see docs, i don't test that variant): aggregate[all-1]: net, ... And there is sense to make aggregation on nfacctd, not on agents. Agents will only collect necessary traffic for nfacctd. Alex > Hi Alex and thank you very much for the reply. I went through your >posting carefully and experimented with your configurations, but I can't >seem to make this work. The flow coming out of the nfprobe plugin still >seems to indicate that all traffic is inbound. Were you able to get >pmacctd to generate either netflow or sflow that reports both in and out >traffic? > > Thanks... > Jim > >> See my letter from 23 Apr 2008 with theme 'best practice (additional >> examples)'. It have attached my personal config. >> >> Alex >> >> >>> Hi All... >>> >>> I'm wondering if there are any docs or samples for configuring the >>> netflow and/or sflow plugins, beyond what accompanies the distribution >>> of pmacct? I'm still having the issue below. >>> >>> It seems that the plugins combine the in and out data together, and the >>> flow collector can not separate it. I must be configuring it wrong. >>> Here is the conf I have been experimenting with: >>> >>> ! pmacctd configuration >>> ! >>> ! >>> ! >>> daemonize: false >>> !debug: true >>> interface: eth0 >>> plugins: nfprobe[out], nfprobe[in] >>> !plugins: print[in], print[out] >>> !plugins: memory[in], memory[out] >>> ! >>> aggregate[in]: src_host, src_port, dst_host, dst_port, proto, flows, tos >>> !aggregate_filter[in]: net 72.46.0.0/17 >>> nfprobe_receiver[in]: 72.46.65.58:9996 >>> nfprobe_version: 9 >>> ! >>> aggregate[out]: src_host, src_port, dst_host, dst_port, proto, flows, tos >>> !aggregate_filter[out]: net 72.46.0.0/17 >>> nfprobe_receiver[out]: 72.46.65.58:9996 >>> nfprobe_version: 9 >>> ! >>> >>> Any hints, please? ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
Hi Alex and thank you very much for the reply. I went through your posting carefully and experimented with your configurations, but I can't seem to make this work. The flow coming out of the nfprobe plugin still seems to indicate that all traffic is inbound. Were you able to get pmacctd to generate either netflow or sflow that reports both in and out traffic? Thanks... Jim --On Wednesday, May 14, 2008 2:39 PM +0300 alex <[EMAIL PROTECTED]> wrote: > See my letter from 23 Apr 2008 with theme 'best practice (additional > examples)'. It have attached my personal config. > > Alex > > >> Hi All... >> >> I'm wondering if there are any docs or samples for configuring the >> netflow and/or sflow plugins, beyond what accompanies the distribution >> of pmacct? I'm still having the issue below. >> >> It seems that the plugins combine the in and out data together, and the >> flow collector can not separate it. I must be configuring it wrong. >> Here is the conf I have been experimenting with: >> >> ! pmacctd configuration >> ! >> ! >> ! >> daemonize: false >> !debug: true >> interface: eth0 >> plugins: nfprobe[out], nfprobe[in] >> !plugins: print[in], print[out] >> !plugins: memory[in], memory[out] >> ! >> aggregate[in]: src_host, src_port, dst_host, dst_port, proto, flows, tos >> !aggregate_filter[in]: net 72.46.0.0/17 >> nfprobe_receiver[in]: 72.46.65.58:9996 >> nfprobe_version: 9 >> ! >> aggregate[out]: src_host, src_port, dst_host, dst_port, proto, flows, tos >> !aggregate_filter[out]: net 72.46.0.0/17 >> nfprobe_receiver[out]: 72.46.65.58:9996 >> nfprobe_version: 9 >> ! >> >> Any hints, please? > > > > На этап Кубка мира BelSwissBank летят даже > частными самолетами. > http://news.tut.by/society/108880.html > > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
See my letter from 23 Apr 2008 with theme 'best practice (additional examples)'. It have attached my personal config. Alex > Hi All... > > I'm wondering if there are any docs or samples for configuring the netflow > and/or sflow plugins, beyond what accompanies the distribution of pmacct? > I'm still having the issue below. > > It seems that the plugins combine the in and out data together, and the > flow collector can not separate it. I must be configuring it wrong. Here > is the conf I have been experimenting with: > > ! pmacctd configuration > ! > ! > ! > daemonize: false > !debug: true > interface: eth0 > plugins: nfprobe[out], nfprobe[in] > !plugins: print[in], print[out] > !plugins: memory[in], memory[out] > ! > aggregate[in]: src_host, src_port, dst_host, dst_port, proto, flows, tos > !aggregate_filter[in]: net 72.46.0.0/17 > nfprobe_receiver[in]: 72.46.65.58:9996 > nfprobe_version: 9 > ! > aggregate[out]: src_host, src_port, dst_host, dst_port, proto, flows, tos > !aggregate_filter[out]: net 72.46.0.0/17 > nfprobe_receiver[out]: 72.46.65.58:9996 > nfprobe_version: 9 > ! > > Any hints, please? На этап Кубка мира BelSwissBank летят даже частными самолетами. http://news.tut.by/society/108880.html ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Multiple interfaces and netflow
Hi All... I'm wondering if there are any docs or samples for configuring the netflow and/or sflow plugins, beyond what accompanies the distribution of pmacct? I'm still having the issue below. It seems that the plugins combine the in and out data together, and the flow collector can not separate it. I must be configuring it wrong. Here is the conf I have been experimenting with: ! pmacctd configuration ! ! ! daemonize: false !debug: true interface: eth0 plugins: nfprobe[out], nfprobe[in] !plugins: print[in], print[out] !plugins: memory[in], memory[out] ! aggregate[in]: src_host, src_port, dst_host, dst_port, proto, flows, tos !aggregate_filter[in]: net 72.46.0.0/17 nfprobe_receiver[in]: 72.46.65.58:9996 nfprobe_version: 9 ! aggregate[out]: src_host, src_port, dst_host, dst_port, proto, flows, tos !aggregate_filter[out]: net 72.46.0.0/17 nfprobe_receiver[out]: 72.46.65.58:9996 nfprobe_version: 9 ! Any hints, please? --On Saturday, May 10, 2008 4:38 AM -0400 Jim Archer <[EMAIL PROTECTED]> wrote: > Hi All... > > I am trying to use pmacct on a Vyatta router to generate netflow, to be > collected by Netflow Analyzer by ManageEngine. The operating system is > Debian Lenny. > > I installed the pmacct package from Lenny and had no trouble setting it > up. I used the Debian method of setting the environment variable to > "eth0 eth1 eth2" because those are the interfaces I want to monitor. I > provided a conf file for each interface, names appropriately. > > Staring pmacct using the script in init.d results in 6 processes, > apparently two for each interface. Also, netstat shows three udp > sessions to the IP that the Netflow app resides on, on the correct port. > > But there is a problem. Netflow shows that there is data but only from > one interface. It shows data in only, none out. If I use a Cisco 7200 > to generate netflow, Netflow Analyzer shows all the interfaces in each > flow. > > I have checked a bunch of things, all with no luck. Does anyone have any > idea what I might have done wrong? > > Thanks very much... > > Jim > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] Multiple interfaces and netflow
Hi All... I am trying to use pmacct on a Vyatta router to generate netflow, to be collected by Netflow Analyzer by ManageEngine. The operating system is Debian Lenny. I installed the pmacct package from Lenny and had no trouble setting it up. I used the Debian method of setting the environment variable to "eth0 eth1 eth2" because those are the interfaces I want to monitor. I provided a conf file for each interface, names appropriately. Staring pmacct using the script in init.d results in 6 processes, apparently two for each interface. Also, netstat shows three udp sessions to the IP that the Netflow app resides on, on the correct port. But there is a problem. Netflow shows that there is data but only from one interface. It shows data in only, none out. If I use a Cisco 7200 to generate netflow, Netflow Analyzer shows all the interfaces in each flow. I have checked a bunch of things, all with no luck. Does anyone have any idea what I might have done wrong? Thanks very much... Jim ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
