Re: [pmacct-discussion] Broken aggregate Filter
Hi Bernd, You appear to be using VLANs - please try amending your 'aggregate_filter' lines in the following fashion: 'vlan and existing statement'. Let me know whether that solves the issue. Cheers, Paolo On Wed, Jun 08, 2011 at 10:26:57AM +, Bernd Bornkessel wrote: Hi, we want to you sfacctd for accounting and billing purposes in our network. I set up a test installation with a router sending SFlow and a Debian 6.0 system with pmacct-0.14.0rc1 and a mysql backend as the collector. Everything works fine as long as I don't use the aggregate filters. Without aggregate filtering sfacctd will write the data to the database. But as we need to have filters for our own networks, I tried to use the aggregate filters. But as soon as I activate them, there will be no data written to the database. The problem exists with all backends (mysql, memory, sqlite etc.). Thus I assume the filters not matching. But according to the sent flows and the configured filters, there should be flow data afer filtering. [ ... ] ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Broken aggregate Filter
Hi Bernd, On Thu, 9 Jun 2011, Bernd Bornkessel wrote: It works if I use: vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) Well, but what if I also want to filter by VLAN. The following filters do not work :\ [...[ vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) These filters look identical to me. How come it both works and doesn't work? Cheers, Chris. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Broken aggregate Filter
Hi Chris, Sorry, just a copy-pasty-mistake. The working filter is: vlan and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19) The non-working are: vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) Cheers, Bernd -Ursprüngliche Nachricht- Von: pmacct-discussion-boun...@pmacct.net [mailto:pmacct-discussion- boun...@pmacct.net] Im Auftrag von Chris Wilson Gesendet: Donnerstag, 9. Juni 2011 18:46 An: pmacct-discussion@pmacct.net Betreff: Re: [pmacct-discussion] Broken aggregate Filter Hi Bernd, On Thu, 9 Jun 2011, Bernd Bornkessel wrote: It works if I use: vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) Well, but what if I also want to filter by VLAN. The following filters do not work :\ [...[ vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) These filters look identical to me. How come it both works and doesn't work? Cheers, Chris. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Broken aggregate Filter
Hi Bernd, On Thu, 9 Jun 2011, Bernd Bornkessel wrote: The working filter is: vlan and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19) The non-working are: vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) I think you may be falling victim to this (from man pcap-filter(7)): vlan [vlan_id] True if the packet is an IEEE 802.1Q VLAN packet. If [vlan_id] is specified, only true if the packet has the specified vlan_id. Note that the first vlan keyword encountered in expression changes the decoding offsets for the remainder of expression on the assumption that the packet is a VLAN packet. The vlan [vlan_id] expression may be used more than once, to filter on VLAN hierarchies. Each use of that expression increments the filter offsets by 4. Therefore I don't think you can use the vlan keyword more than once in the same expression (unless you have vlan hierarchies). This appears to be a limitation (and a rather unusual one) of libpcap, not pmacct. If they really want to support nested vlans (and I would seriously question the sanity of anyone who used them) I would respectfully suggest that they modify the vlan keyword to not change the filter offset, and create a new keyword like nested-vlan which does. Cheers, Chris. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
Re: [pmacct-discussion] Broken aggregate Filter
Hi Bernd, Every incoming sflow sample should have the output interface ifindex associated with it and you can add that to your aggregation primitives in pmacct. Would it not be possible to just query the pmacct DB for an associated list of ISP/Egress ifindexes? If you want to look at ISP/Ingress you could do something similar with the ingress ifindex. -Brent On Jun 9, 2011, at 1:07 PM, Bernd Bornkessel wrote: Hi Chris, thanks. So vlan based filtering will not work with more than one vlan. I'm thinking about a workaround. Something about the background: We're using Arista 7120 switches in one of our DC locations. These switches mainly build the 10G layer-2 infrastructure for our vSphere environment connecting the VMware servers as well as an iSCSI storage. The second purpose for the switches is acting as our core routers. The problem is, that we only need to account traffic that is been routed to the ISP uplinks. Unfortunately the sflow implementation for this switches is ingress only per each physical interface. Thus we need to activate sflow for each interface and filter out the required flows. So I need to filter for the vlans and our public networks. Two workarounds came to my mind. 1. We do not filter for the vlan, but use the vlan for aggregation instead. On a daily basis we remove the records for unneccessary vlans from the database. 2. I create an incoming and outgoing plugin/aggregate/filter for each vlan that aggregate and write to the database independently. Is there a recommendation for a maximum plugin instances? We need at least 6 vlans resulting in 12 instances. Cheers, Bernd -Ursprüngliche Nachricht- Von: pmacct-discussion-boun...@pmacct.net [mailto:pmacct-discussion- boun...@pmacct.net] Im Auftrag von Chris Wilson Gesendet: Donnerstag, 9. Juni 2011 19:04 An: pmacct-discussion@pmacct.net Betreff: Re: [pmacct-discussion] Broken aggregate Filter Hi Bernd, On Thu, 9 Jun 2011, Bernd Bornkessel wrote: The working filter is: vlan and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19) The non-working are: vlan and ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) ((vlan 365 or vlan 1337) and (dst net 192.76.141.0/24 or dst net 194.55.246.0/23 or dst net 195.246.160/19 or dst net 88.215.224.0/19 or dst net 62.93.212.0/23 or dst net 62.93.246.0/23 or dst net 88.215.192.0/19)) I think you may be falling victim to this (from man pcap-filter(7)): vlan [vlan_id] True if the packet is an IEEE 802.1Q VLAN packet. If [vlan_id] is specified, only true if the packet has the specified vlan_id. Note that the first vlan keyword encountered in expression changes the decoding offsets for the remainder of expression on the assumption that the packet is a VLAN packet. The vlan [vlan_id] expression may be used more than once, to filter on VLAN hierarchies. Each use of that expression increments the filter offsets by 4. Therefore I don't think you can use the vlan keyword more than once in the same expression (unless you have vlan hierarchies). This appears to be a limitation (and a rather unusual one) of libpcap, not pmacct. If they really want to support nested vlans (and I would seriously question the sanity of anyone who used them) I would respectfully suggest that they modify the vlan keyword to not change the filter offset, and create a new keyword like nested-vlan which does. Cheers, Chris. ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
