Re: [pmacct-discussion] nfacctd pretag.map with mpls_vpn_rd
Hi Paolo, Thanks for the explanation and talking through this with me. Indeed flow_to_rd.map was the missing piece I needed. I have things working the way we wanted by generating a flow_to_rd.map that matches on the router_ip and mpls_vpn_id. Your SNMP suggestion would also work for us too since we have all that info readily available, but going the mpls_vpn_id route keeps the number of entries smaller and should change less. I had previously played around with flow_to_rd.map but for whatever reason convinced myself it wasn’t affecting the path lookups. It is very clearly working though now, no many thanks for your guidance. >From my perspective issue 770 can also be closed. Thanks, Andy On March 19, 2024 at 4:33:08 AM, Paolo Lucente (pa...@pmacct.net) wrote: Hi Andy, Amazing! So probably, after all, we may bash Issue 770 on GitHub -- coolio. Let me recap situation so that you can correct me where i am wrong and fill any gaps. You have flows, these (all or some) have a VRFID; you have BGP feeds and there, obviously, you have MPLS VPN RD. The linking pin is the flow_to_rd_map (see examples/flow_to_rd.map.example and CONFIG-KEYS). The map allows you to work in a few modes: 'id' is the output, the RD as you want to match it in the BGP feed; you can output the RD in a few different modes: basing on router / interface, for example; or VRFID; or the MPLS label stack. Probably you want the VRFID scenario. Problem that often people come across is .. what is the VRFID anyway, where do i source the info? Of course some vendor-specific SNMP polling; so some people i have seen preferring to go to the router / interface scenario, because if you have some sort of inventory / source of truth that is easy to compose. Probably the least popular scenario is composing the map with MPLS labels but now in the software-defined world, potentially with controllers, that sporadically happens too. Paolo On 19/3/24 05:57, Andrew Lake wrote: > Hi Paolo, > > Ok maybe I have gotten headed down the wrong path. It sounds like you’re > saying nfacctd in normal collector mode should be able to take into > account the VRF when asking the BGP daemon to lookup the AS path for a > flow? This would be my ideal situation, and the replication setup I had > was just trying to get around the fact that I didn’t think it did based > on some preliminary testing. I might have given up too soon when it > didn’t appear to be working though. > > Are there any extra options in nfacctd.conf or similar I need to set to > make it take the VRF into consideration for the path lookup? > > What fields does it look at in the BGP message and IPFIX messages to > make the decision? rd and mpls_vpn_rd respectively? Mainly just asking > so I can debug if something was not getting set properly by our routers > in either of the message sets when I was testing. > > > Thanks again, > Andy > > On March 16, 2024 at 1:06:48 AM, Paolo Lucente (pa...@pmacct.net > <mailto:pa...@pmacct.net>) wrote: > >> >> Hi Andy, >> >> Thanks for opening the issue on GitHub and the kind words. >> >> Thing is all you want to achieve is supported in pmacct when working in >> collector mode where the proper inspection of each flow is performed. >> >> Why don't you leave the tee part barebone and implement these features >> in the collector? Just an idea as a perfectly valid answer could be that >> performing this enrichment in the replicator then also 3rd party tools >> could benefit from this info. Let me know. >> >> Paolo >> >> >> On 16/3/24 01:51, Andrew Lake wrote: >> > Hi Paolo, >> > >> > Thanks for reply! I have created the issue as requested: >> > https://github.com/pmacct/pmacct/issues/770 >> <https://github.com/pmacct/pmacct/issues/770> >> > <https://github.com/pmacct/pmacct/issues/770 >> <https://github.com/pmacct/pmacct/issues/770>>. Apologies for missing the >> > docs about tee/replication mode and what you described make sense. >> > >> > Your comment about the matching RD in the BGP messages I think is >> > somewhat related to my ultimate goal, which is to enrich the IPFIX >> > records with the AS Path using the BGP table for the matching VRF. >> > Balancing this with the fact that our routers have limitations in the >> > number of addresses to which they will forward IPFIX, my plan was to >> > have pmacct in tee mode forward the IPFIX records to a pmacct instance >> > dedicated to peering with only one VRF where it could do the BGP lookup. >> > Alternative would be to try to lookup up the path by mapping the IPFIX >> > VRF ID to the BGP RD and then basing the lookup on that value in >> &
Re: [pmacct-discussion] nfacctd pretag.map with mpls_vpn_rd
Hi Paolo, Ok maybe I have gotten headed down the wrong path. It sounds like you’re saying nfacctd in normal collector mode should be able to take into account the VRF when asking the BGP daemon to lookup the AS path for a flow? This would be my ideal situation, and the replication setup I had was just trying to get around the fact that I didn’t think it did based on some preliminary testing. I might have given up too soon when it didn’t appear to be working though. Are there any extra options in nfacctd.conf or similar I need to set to make it take the VRF into consideration for the path lookup? What fields does it look at in the BGP message and IPFIX messages to make the decision? rd and mpls_vpn_rd respectively? Mainly just asking so I can debug if something was not getting set properly by our routers in either of the message sets when I was testing. Thanks again, Andy On March 16, 2024 at 1:06:48 AM, Paolo Lucente (pa...@pmacct.net) wrote: Hi Andy, Thanks for opening the issue on GitHub and the kind words. Thing is all you want to achieve is supported in pmacct when working in collector mode where the proper inspection of each flow is performed. Why don't you leave the tee part barebone and implement these features in the collector? Just an idea as a perfectly valid answer could be that performing this enrichment in the replicator then also 3rd party tools could benefit from this info. Let me know. Paolo On 16/3/24 01:51, Andrew Lake wrote: > Hi Paolo, > > Thanks for reply! I have created the issue as requested: > https://github.com/pmacct/pmacct/issues/770 > <https://github.com/pmacct/pmacct/issues/770>. Apologies for missing the > docs about tee/replication mode and what you described make sense. > > Your comment about the matching RD in the BGP messages I think is > somewhat related to my ultimate goal, which is to enrich the IPFIX > records with the AS Path using the BGP table for the matching VRF. > Balancing this with the fact that our routers have limitations in the > number of addresses to which they will forward IPFIX, my plan was to > have pmacct in tee mode forward the IPFIX records to a pmacct instance > dedicated to peering with only one VRF where it could do the BGP lookup. > Alternative would be to try to lookup up the path by mapping the IPFIX > VRF ID to the BGP RD and then basing the lookup on that value in > addition to the prefix, but this seems like non-hanging fruit as you > said. if you are able to get the VRF ID matching against IPFIX working > so we can tee it from there, that will be fantastic. > > Thanks again for all your help…and also just in general building an > awesome product :) > > Andy > > On March 15, 2024 at 2:18:01 AM, Paolo Lucente (pa...@pmacct.net > <mailto:pa...@pmacct.net>) wrote: > >> >> Hi Andy, >> >> mpls_vpn_rd is supported in pre_tag_map however it is not supported when >> in tee / replication mode (this is documented). >> >> For your specific use-case, since you are interested in matching the VRF >> ID, which in turn is self-consistent as part of an IPFIX record, this is >> something that can be achieved. However this may then open the door to >> somebody wanting to match the RD for a prefix as coming from a BGP / BMP >> feed, and for this a few (non-hanging-fruit) steps would be needed; then >> again the limitation to the self-contained VRF ID scenario can be >> documented. >> >> May i ask you to open an Issue on GitHub? I'll flag it as Enhancement >> right away and will be able to make progress pretty soon for the VRF ID >> case. >> >> Paolo >> >> >> On 13/3/24 03:11, Andrew Lake wrote: >> > Hi, >> > >> > I recently tried creating a setup where I have an instance of nfacctd >> > running that has a pretag.map that I want to look at the value of >> > mpls_vpn_rd as determined from the IPFIX record and then set a tag. The >> > plan is to then use the tee plugin to send the IPFIX traffic to a >> > different nfacctd instance based on the tag set. I can get more into why >> > we’re doing this if interested, but I ran into a snag that I can’t seem >> > to figure out on my own. nfacctd doesn’t seems to like when I add >> > mpls_vpn_rd to the pretag.map. I get messages like: >> > >> > [/etc/pmacct/pretag.map:3] unknown key 'mpls_vpn_rd'. Ignored. >> > >> > The pretag.map is pretty vanilla. Just lines like: >> > >> > set_tag=1 mpls_vpn_rd= >> > >> > My nfacctd.conf: >> > >> > ! Port where nfacctd listens >> > nfacctd_port: 9996 >> > >> > ! Adds debugging output to logs. Disable in production. >>
Re: [pmacct-discussion] nfacctd pretag.map with mpls_vpn_rd
Hi Paolo, Thanks for reply! I have created the issue as requested: https://github.com/pmacct/pmacct/issues/770. Apologies for missing the docs about tee/replication mode and what you described make sense. Your comment about the matching RD in the BGP messages I think is somewhat related to my ultimate goal, which is to enrich the IPFIX records with the AS Path using the BGP table for the matching VRF. Balancing this with the fact that our routers have limitations in the number of addresses to which they will forward IPFIX, my plan was to have pmacct in tee mode forward the IPFIX records to a pmacct instance dedicated to peering with only one VRF where it could do the BGP lookup. Alternative would be to try to lookup up the path by mapping the IPFIX VRF ID to the BGP RD and then basing the lookup on that value in addition to the prefix, but this seems like non-hanging fruit as you said. if you are able to get the VRF ID matching against IPFIX working so we can tee it from there, that will be fantastic. Thanks again for all your help…and also just in general building an awesome product :) Andy On March 15, 2024 at 2:18:01 AM, Paolo Lucente (pa...@pmacct.net) wrote: Hi Andy, mpls_vpn_rd is supported in pre_tag_map however it is not supported when in tee / replication mode (this is documented). For your specific use-case, since you are interested in matching the VRF ID, which in turn is self-consistent as part of an IPFIX record, this is something that can be achieved. However this may then open the door to somebody wanting to match the RD for a prefix as coming from a BGP / BMP feed, and for this a few (non-hanging-fruit) steps would be needed; then again the limitation to the self-contained VRF ID scenario can be documented. May i ask you to open an Issue on GitHub? I'll flag it as Enhancement right away and will be able to make progress pretty soon for the VRF ID case. Paolo On 13/3/24 03:11, Andrew Lake wrote: > Hi, > > I recently tried creating a setup where I have an instance of nfacctd > running that has a pretag.map that I want to look at the value of > mpls_vpn_rd as determined from the IPFIX record and then set a tag. The > plan is to then use the tee plugin to send the IPFIX traffic to a > different nfacctd instance based on the tag set. I can get more into why > we’re doing this if interested, but I ran into a snag that I can’t seem > to figure out on my own. nfacctd doesn’t seems to like when I add > mpls_vpn_rd to the pretag.map. I get messages like: > > [/etc/pmacct/pretag.map:3] unknown key 'mpls_vpn_rd'. Ignored. > > The pretag.map is pretty vanilla. Just lines like: > > set_tag=1 mpls_vpn_rd= > > My nfacctd.conf: > > ! Port where nfacctd listens > nfacctd_port: 9996 > > ! Adds debugging output to logs. Disable in production. > debug: true > > ! tag flow values to determine where tee sends them next > pre_tag_map: /etc/pmacct/pretag.map > > plugins: tee > tee_receivers: /etc/pmacct/tee_receivers.lst > tee_transparent: true > > > > I went so far as to copy the exact mpls_vpn_rd line from the example in > the git repo just to see if it would accept it and it still complained. > From looking at the documentation looks like mpls_vpn_rd should be > allowed in nfacctd (I think?). I tried following the code path in the > source but was having trouble telling which “Ignored” message was > getting triggered, and figured maybe I was better off just asking before > I got too far down the rabbit hole. > > Is a pretag.map with mpls_vpn_rd supported by nfacctd? If yes, any ideas > where to look next? Or any other info I should send? > > Thanks, > Andy > > > > > ___ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
[pmacct-discussion] nfacctd pretag.map with mpls_vpn_rd
Hi, I recently tried creating a setup where I have an instance of nfacctd running that has a pretag.map that I want to look at the value of mpls_vpn_rd as determined from the IPFIX record and then set a tag. The plan is to then use the tee plugin to send the IPFIX traffic to a different nfacctd instance based on the tag set. I can get more into why we’re doing this if interested, but I ran into a snag that I can’t seem to figure out on my own. nfacctd doesn’t seems to like when I add mpls_vpn_rd to the pretag.map. I get messages like: [/etc/pmacct/pretag.map:3] unknown key 'mpls_vpn_rd'. Ignored. The pretag.map is pretty vanilla. Just lines like: set_tag=1 mpls_vpn_rd= My nfacctd.conf: ! Port where nfacctd listens nfacctd_port: 9996 ! Adds debugging output to logs. Disable in production. debug: true ! tag flow values to determine where tee sends them next pre_tag_map: /etc/pmacct/pretag.map plugins: tee tee_receivers: /etc/pmacct/tee_receivers.lst tee_transparent: true I went so far as to copy the exact mpls_vpn_rd line from the example in the git repo just to see if it would accept it and it still complained. From looking at the documentation looks like mpls_vpn_rd should be allowed in nfacctd (I think?). I tried following the code path in the source but was having trouble telling which “Ignored” message was getting triggered, and figured maybe I was better off just asking before I got too far down the rabbit hole. Is a pretag.map with mpls_vpn_rd supported by nfacctd? If yes, any ideas where to look next? Or any other info I should send? Thanks, Andy ___ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
