bgpd simplify kroute nexthop handling

2022-07-26 Thread Claudio Jeker
This is another step in the epic kroute rework. Interfaces (kif) come with a list of kroutes attached to them which are only used to track the interface state and to fiddle with nexthop states. Now these lists are not really needed. One can just validate the nexthops without losing any relevant

bgpd more IPv6 scope_id love

2022-07-14 Thread Claudio Jeker
Noticed while syncing code. I missed some scope_id assignments and checks in a few places. - VPN6 missed all of it - in kr_redistribute6() the copy is not really needed since link local address can not be redistributed (I still added it though) - kroute6_compare() do actually compare the

OpenBGPD 7.5 released

2022-07-15 Thread Claudio Jeker
We have released OpenBGPD 7.5, which will be arriving in the OpenBGPD directory of your local OpenBSD mirror soon. This release includes the following changes to the previous release: * Implement RFC 9234 - Route Leak Prevention and Detection Using Roles in UPDATE and OPEN Messages.

Re: bgpd simplify kroute nexthop handling

2022-07-26 Thread Claudio Jeker
On Tue, Jul 26, 2022 at 05:35:47PM +0200, Theo Buehler wrote: > On Tue, Jul 26, 2022 at 05:17:23PM +0200, Claudio Jeker wrote: > > On Tue, Jul 26, 2022 at 03:51:40PM +0200, Theo Buehler wrote: > > > On Tue, Jul 26, 2022 at 03:09:37PM +0200, Claudio Jeker wrote: > >

adjust bgpctl show fib formatting

2022-07-28 Thread Claudio Jeker
This adjusts the output of bgpctl show fib. It removes the F_DOWN check since kroutes no longer track this. And it changes the flag printing code to reserve the space needed so that adjusting the flags does not break the output. Last but not least increase the size of destination and gateway to

bgpd remove F_DOWN flag

2022-07-28 Thread Claudio Jeker
When the bgpctl show fib diff is committed nothing uses F_DOWN anymore. Remove the flag and reshuffle some of the other flags to group them a bit better. -- :wq Claudio Index: bgpd.h === RCS file: /cvs/src/usr.sbin/bgpd/bgpd.h,v

Re: bgpd more kroute refactor

2022-07-28 Thread Claudio Jeker
On Thu, Jul 28, 2022 at 03:09:18PM +0200, Theo Buehler wrote: > On Thu, Jul 28, 2022 at 12:48:05PM +0200, Claudio Jeker wrote: > > Next step on the epic saga of cleaning up kroute.c > > > > Refactor kroute_remove() so that a struct kroute_full can be passed to the >

bgpd more kroute refactor

2022-07-28 Thread Claudio Jeker
Next step on the epic saga of cleaning up kroute.c Refactor kroute_remove() so that a struct kroute_full can be passed to the function. It updates the struct kroute_full with the route that got removed. I split the code into kroute[46]_remove() to make kroute_remove() less cluttered. The return

bgpd kroute F_KERNEL flag

2022-07-22 Thread Claudio Jeker
There is no need to use F_KERNEL to tag routes from the kernel. All this can be done by priority (RTP_MINE vs anything else). The conversion is simple in most cases. In kr_fib_delete() and kr_fib_change() check if the route is a bgpd owned route and in that case remove the F_BGPD_INSERTED flag.

Re: nd6: Zap nd6_recalc_reachtm_interval indirection

2022-07-22 Thread Claudio Jeker
On Fri, Jul 22, 2022 at 12:18:34PM +, Klemens Nanni wrote: > Only used once, so use the macro directly like ND6_SLOWTIMER_INTERVAL > is used in many places. > > OK? Is that a value that should be adjustable? > diff --git a/sys/netinet6/nd6.c b/sys/netinet6/nd6.c > index

bgpd nexthop check

2022-07-23 Thread Claudio Jeker
Change the logic and name of bgpd_filternexthop(). This function applies the 'nexthop qualify via' config setting. Instead of telling if the route is filtered (true) or not (false) flip the logic around and rename the function to bgpd_oknexthop(). Also flip the internal logic around to simplify

bgpd another try at F_KERNEL removal

2022-07-27 Thread Claudio Jeker
My last try was not successful because kr_tofull() did not return RTP_MINE and so some checks for RTP_MINE instead of F_KERNEL did not work. This diff does two things. It replaces the F_KERNEL checks with !F_BGPD checks. F_KERNEL and F_BGPD are mutual exclusive. On top of that keep kr->priority

Re: bgpd another try at F_KERNEL removal

2022-07-27 Thread Claudio Jeker
On Wed, Jul 27, 2022 at 06:36:22PM +0200, Theo Buehler wrote: > On Wed, Jul 27, 2022 at 05:41:11PM +0200, Claudio Jeker wrote: > > My last try was not successful because kr_tofull() did not return RTP_MINE > > and so some checks for RTP_MINE instead of F_KERNEL did not work. >

Re: bgpd simplify kroute nexthop handling

2022-07-26 Thread Claudio Jeker
On Tue, Jul 26, 2022 at 03:51:40PM +0200, Theo Buehler wrote: > On Tue, Jul 26, 2022 at 03:09:37PM +0200, Claudio Jeker wrote: > > This is another step in the epic kroute rework. > > > > Interfaces (kif) come with a list of kroutes attached to them which are > > only

bgpd decision process and bad dmetric

2022-07-16 Thread Claudio Jeker
I deployed bgpd on one of more core routers and triggered the fatal "bad dmetric in decision process" from time to time. I realized after a longer debugging session that one reason this happens is when nexthops become valid. The state change affects all prefixes at once but then they are

Re: ifconfig description for wireguard peers

2022-07-14 Thread Claudio Jeker
On Thu, Jul 14, 2022 at 10:51:42AM +0200, Stefan Sperling wrote: > On Wed, Jul 13, 2022 at 05:13:49PM +, Mikolaj Kucharski wrote: > > On Wed, Jul 13, 2022 at 05:43:59PM +0100, Stuart Henderson wrote: > > > > > > > > Not sure how to handle long output in different way. If you don't > > > >

Re: vsw.4: mention veb next to bridge

2022-07-21 Thread Claudio Jeker
On Wed, Jul 20, 2022 at 05:27:51PM -0700, Chris Cappuccio wrote: > Klemens Nanni [k...@openbsd.org] wrote: > > veb(4) works just fine in this setup, so don't give the impression only > > bridge(4) would work. > > > > In related items, is it time to tedu bridge(4) and vether(4) ? Is there >

bgpd, more IPv6 scope_id insanity

2022-06-25 Thread Claudio Jeker
Since 128bit address space is not enough add the extra 32bit scope_id to struct kroute6 so that we can track that as well. With this using link local addresses as nexthops should work again. Also the madness is a little bit less bad since we just copy the scope_id in and out no matter what. --

bgplgd - a JSON frontend to bgpd

2022-06-28 Thread Claudio Jeker
2022 14:43:39 - @@ -0,0 +1,179 @@ +.\" $OpenBSD$ +.\" +.\" Copyright (c) 2021 Claudio Jeker +.\" +.\" Permission to use, copy, modify, and distribute this software for any +.\" purpose with or without fee is hereby granted, provided that the above +.\" copyrigh

Re: allow 240/4 in various network daemons

2022-05-06 Thread Claudio Jeker
On Thu, May 05, 2022 at 11:37:24AM +0200, Claudio Jeker wrote: > So most routing daemons and other network daemons like pppd do not allow > 240/4 as IPs because they check the IP against IN_BADCLASS(). > I think it is time to remove this restriction. > > Now there is another

Re: router timer mutex

2022-04-28 Thread Claudio Jeker
On Thu, Apr 28, 2022 at 07:24:22PM +0200, Alexander Bluhm wrote: > I still need an ok for this diff. It is the final step before we > can run IP forwaring in parallel. Fine with me. If it holds you back put it in OK claudio@ I will rip the rttimer code appart in the next days and make that API a

Re: cleanup multicast rttimer queues

2022-04-28 Thread Claudio Jeker
On Thu, Apr 28, 2022 at 05:51:57PM +0200, Alexander Bluhm wrote: > On Wed, Apr 27, 2022 at 12:10:59PM +0200, Claudio Jeker wrote: > > There is no need to have a rttimer queue per rdomain. The rttimer itself > > is rdomain aware and so this just make everything more complicated

simplify rttimer api

2022-04-29 Thread Claudio Jeker
The callback currently uses struct rttimer as an argument but the code only needs the rtt_tableid element from there. Change the callbacks to be of the form void (*rtt_callback)(struct rtentry *r, u_int rtableid) Also change the default rttimer callback (in case the function is NULL) to only

simplify spec_close()

2022-04-27 Thread Claudio Jeker
spec_close() does a fair bit of extra work around the xlock flag. Now spec_close() is called via VOP_CLOSE and either the vnode has VXLOCK set (because it is called via vclean()) or it does not. In either case VXLOCK can not change during call (at least that was my conclusion looking at the code).

Re: [External] : Re: add sanity checks to IGMP/MLD

2022-05-03 Thread Claudio Jeker
On Tue, May 03, 2022 at 02:08:33PM +0200, Alexandr Nedvedicky wrote: > Hello > > On Tue, May 03, 2022 at 10:44:48AM +0200, Claudio Jeker wrote: > > > > > The RFC does not use the usual MUST to enforce any of this. > > So yes, we should probably not be too

rttimer move callback to the queue struct

2022-05-03 Thread Claudio Jeker
Next stage of rttimer cleanup. Move the callback from the rttimer to the rttimer_queue struct. The callback is always the same for a queue so there is no need to define it on every call. On top of that replace rt_timer_queue_destroy() with rt_timer_queue_flush(). With this queues can no longer be

Re: add sanity checks to IGMP/MLD

2022-05-03 Thread Claudio Jeker
On Tue, May 03, 2022 at 12:05:06AM +0200, Alexander Bluhm wrote: > On Mon, May 02, 2022 at 11:30:58PM +0200, Alexandr Nedvedicky wrote: > > hello, > > > > bluhm@ has committed a fix [1] which makes pf to accept IGMP/MLD messages. > > If I remember correct pf(4) was dropping those messages because

Re: ratecheck mutex

2022-05-04 Thread Claudio Jeker
On Wed, May 04, 2022 at 12:14:01AM +0200, Alexander Bluhm wrote: > Hi, > > We have one comment that locking for ratecheck(9) is missing. In > all other places locking status of the struct timeval *lasttime > is unclear. > > The easiest fix is a global mutex for all lasttime in ratecheck(). >

Re: rpki-client: don't time out in offline mode

2022-05-04 Thread Claudio Jeker
On Wed, May 04, 2022 at 03:51:02PM +0200, Theo Buehler wrote: > I had output from rpki-client -f something piped into less. After an > hour rpki-client couldn't take it any longer and decided to move on to > a better place. It also left a residue via syslog on its way out. I > don't think it

Re: rpki-client: don't time out in offline mode

2022-05-04 Thread Claudio Jeker
On Wed, May 04, 2022 at 04:09:41PM +0200, Theo Buehler wrote: > On Wed, May 04, 2022 at 04:03:21PM +0200, Claudio Jeker wrote: > > On Wed, May 04, 2022 at 03:51:02PM +0200, Theo Buehler wrote: > > > I had output from rpki-client -f something piped into less. After an >

Re: rpki-client: deserialize IPAddrBlocks in libcrypto

2022-05-10 Thread Claudio Jeker
On Tue, May 10, 2022 at 01:27:17PM +0200, Theo Buehler wrote: > This is a straightforward conversion to letting libcrypto's RFC 3779 > code parse a cert's IPAddrBlocks. The magic happens in X509V3_EXT_d2i() > in sbgp_ipaddrblk(). After that, we simply have to walk the returned > structure. The

Re: rpki-client: three leaks in cert.c

2022-05-10 Thread Claudio Jeker
On Tue, May 10, 2022 at 01:47:44PM +0200, Theo Buehler wrote: > In sbgp_asrange() and sbgp_addr_range(), the ASN1_SEQUENCE_ANY *seq is > potentially leaked due to early return 0 instead of goto out. The last > hunk collides with my IPAddrBlocks diff. Sending this out so I don't > forget. > >

Re: rpki-client: deserialize ASIdentifiers in libcrypto

2022-05-11 Thread Claudio Jeker
On Tue, May 10, 2022 at 08:43:45PM +0200, Theo Buehler wrote: > The ASIdentifiers code is a bit strangely factored presumably due to > constraints of the low-level shoveling. I kept the coarse structure > of the code and left some house keeping for later. The changes in > sbgp_asrange() and

Re: rpki-client: cache X509v3 extensions early

2022-05-11 Thread Claudio Jeker
On Wed, May 11, 2022 at 08:50:57AM -0600, Bob Beck wrote: > yes makes sense > > ok beck@ agreed, ok claudio@ > > On May 11, 2022, at 07:53, Theo Buehler wrote: > > > > Some funky libcrypto business ahead. > > > > X509 API functions such as X509_check_ca() or X509_get_extension_flags() > >

Re: rpki-client: add support for draft-ietf-sidrops-rpki-rsc in filemode

2022-05-09 Thread Claudio Jeker
On Sun, May 08, 2022 at 08:05:08PM +, Job Snijders wrote: > Dear Theo, fellow developers, > > Many thanks for the first review pass, much appreciated. > > > This is a good first step. I have a few initial comments inline. Once you > > fix > > those, review of the rest will be easier. > > >

Re: rpki-client: add support for draft-ietf-sidrops-rpki-rsc in filemode

2022-05-09 Thread Claudio Jeker
On Mon, May 09, 2022 at 12:53:05PM +0200, Theo Buehler wrote: > > As the various same-named-but-different 'parse' structs are not easily > > interchangeable without more refactoring, I marked them "XXX:". Perhaps > > we can work on that in tree? > > I'm fine with fixing that in-tree. Sorry about

Re: rpki-client: add support for draft-ietf-sidrops-rpki-rsc in filemode

2022-05-09 Thread Claudio Jeker
On Mon, May 09, 2022 at 01:07:17PM +, Job Snijders wrote: > On Mon, May 09, 2022 at 12:11:22PM +0200, Claudio Jeker wrote: > > why does the draft allow for optional filenames? What the heck is the > > digest then covering some random gunk? > > Yes, that is entirely

Re: rpki-client: enforce RSC compliant filenames

2022-05-10 Thread Claudio Jeker
On Tue, May 10, 2022 at 08:05:00AM +0200, Theo Buehler wrote: > This moves valid_filename() to validate.c and splits out a helper > portable_filename() which can be used from the RSC code. While moving > valid_filename() is not necessary, I thought it makes sense to keep the > two functions next

Re: rpki-client: enforce RSC compliant filenames

2022-05-10 Thread Claudio Jeker
On Tue, May 10, 2022 at 08:38:32AM +0200, Theo Buehler wrote: > On Tue, May 10, 2022 at 08:28:10AM +0200, Claudio Jeker wrote: > > On Tue, May 10, 2022 at 08:05:00AM +0200, Theo Buehler wrote: > > > This moves valid_filename() to validate.c and splits out a helper > > >

Re: Reserved address behavior (alternate broadcast and 240/4)

2022-05-05 Thread Claudio Jeker
On Thu, May 05, 2022 at 12:58:06PM +1000, Damien Miller wrote: > On Wed, 4 May 2022, Seth David Schoen wrote: > > [snip] > > > Anyway, one thing we would like to propose that OpenBSD update is the > > in_canforward treatment of 240/4 (former class E) addresses. Apparently > > mainly as a result

allow 240/4 in various network daemons

2022-05-05 Thread Claudio Jeker
So most routing daemons and other network daemons like pppd do not allow 240/4 as IPs because they check the IP against IN_BADCLASS(). I think it is time to remove this restriction. Now there is another magical network 0.0.0.0/8 which is not allowed in some but not all of the routing daemons. Not

Use static allocation for rt_timer_queue

2022-05-05 Thread Claudio Jeker
In total there are 6 rt_timer_queues in our kernel. 3 IPv4 and 3 IPv6. That number may be increased to 8 if arp and nd would use these timers as well. Because of this allocation the queue heads via pool(9) is overkill. Switch rt_timer_queue_create to rt_timer_queue_init which just sets up the

more rpki-client refactor

2022-05-11 Thread Claudio Jeker
I took the liberty and refactored the sbgp_assysnum() code a bit more. Main goal is to replace the reallocarray() in append_as() with an upfront calloc() call since now the size is known. Also I decided to collaps sbgp_asnum() into sbgp_assysnum(). One could also inline the now very simple

Re: bgpd force fib sync in fetchtable

2022-08-02 Thread Claudio Jeker
On Tue, Aug 02, 2022 at 01:44:42PM +0200, Theo Buehler wrote: > On Tue, Aug 02, 2022 at 12:34:40PM +0200, Claudio Jeker wrote: > > On startup we load the routing table in bgpd and at that moment a cleanup > > of old bgpd routes should happen. I noticed this is not the case becau

bgpd force fib sync in fetchtable

2022-08-02 Thread Claudio Jeker
On startup we load the routing table in bgpd and at that moment a cleanup of old bgpd routes should happen. I noticed this is not the case because fib_sync is not set and so send_rtmsg() just returns. I think we need to force fib_sync in fetchtable() to make sure the cleanup happens correctly.

Re: rpki-client: disallow inherit in ROA EE IP Resources extension

2022-08-19 Thread Claudio Jeker
On Sat, Aug 13, 2022 at 04:51:05PM +0200, Theo Buehler wrote: > On Fri, Aug 12, 2022 at 09:59:11PM +0200, Theo Buehler wrote: > > On Wed, Aug 10, 2022 at 06:16:30PM +0200, Theo Buehler wrote: > > > On Wed, Aug 10, 2022 at 03:10:19PM +, Job Snijders wrote: > > > > Hi all, > > > > > > > > An

Re: rpki-client: check absence of SIA extension in rsc.c

2022-08-19 Thread Claudio Jeker
On Fri, Aug 19, 2022 at 05:03:30PM +0200, Theo Buehler wrote: > This implements this bit from section 2 (also mentioned in section 5) of > the latest RSC draft: > > because RSCs MUST NOT be distributed through the >global RPKI Repository system, the Subject Information

bgpd silence "connection from non-peer" unless verbose

2022-08-23 Thread Claudio Jeker
I noticed that the "connection from non-peer" message can fill the log and be so chatty that it is hard to see the other messages. The system I see this on is a bit special since it gets hammered by incorrectly configured systems. Maybe other people find this message helpful. If so please speak up

bgpd move nexthop connected magic to kroute

2022-08-23 Thread Claudio Jeker
The RDE does some magic dance around connected networks and their gateway which should be done in kroute.c instead. At least then both functions doing gateway lookups do this magic in the same .c file. It also makes the RDE code a simpler which is good. The RDE actually no longer uses this

Re: rpki-client: retire valid_cert()

2022-08-23 Thread Claudio Jeker
On Mon, Aug 22, 2022 at 12:14:53PM +0200, Theo Buehler wrote: > rpki-client portable makes sure that libcrypto has RFC 3779 support. > Therefore the X509_verify_cert() call in valid_x509() will already > perform the checks that the RFC 3779 extensions are covered along the > chain. While

Re: bgpd silence "connection from non-peer" unless verbose

2022-08-25 Thread Claudio Jeker
On Thu, Aug 25, 2022 at 01:48:50PM +0100, Stuart Henderson wrote: > On 2022/08/25 14:38, Claudio Jeker wrote: > > On Thu, Aug 25, 2022 at 09:23:01AM +0100, Stuart Henderson wrote: > > > On 2022/08/24 18:47, Denis Fondras wrote: > > > > Le Tue, Aug 23, 2022 at 0

Re: rpki-client: add mode to print encapsulated certs/crls in human-readable & PEM format

2022-08-25 Thread Claudio Jeker
file ...\n"); > + " rpki-client [-Vv] [-d cachedir] [-t tal] -f file ...\n" > + " rpki-client -p file ...\n"); > return 1; > } > Index: print.c > === &g

Re: bgpd silence "connection from non-peer" unless verbose

2022-08-25 Thread Claudio Jeker
On Thu, Aug 25, 2022 at 09:23:01AM +0100, Stuart Henderson wrote: > On 2022/08/24 18:47, Denis Fondras wrote: > > Le Tue, Aug 23, 2022 at 06:28:12PM +0200, Claudio Jeker a écrit : > > > I noticed that the "connection from non-peer" message can fill the log and > &

bgplgd use memset and memcpy instead of bzero and bcopy

2022-08-25 Thread Claudio Jeker
The same change was done in bgpd and bgpctl. So here is bgplgd. I replaced one bcopy() with memmove() since this is most probably an overlapping memory move. -- :wq Claudio Index: qs.c === RCS file: /cvs/src/usr.sbin/bgplgd/qs.c,v

Re: bgpd fix peer signaling bug for busy systems

2022-08-26 Thread Claudio Jeker
On Fri, Aug 26, 2022 at 01:42:15PM +0200, Theo Buehler wrote: > On Fri, Aug 26, 2022 at 10:58:38AM +0200, Claudio Jeker wrote: > > Noticed on a route collector with >100 full feeds and well 80Mio prefixes. > > On startup the RDE slurps in a lot of messages and then slowly process

Re: struct ifnet: remove unused if_switchport member

2022-08-26 Thread Claudio Jeker
On Fri, Aug 26, 2022 at 04:15:43PM +0100, Stuart Henderson wrote: > On 2022/08/26 09:49, Klemens Nanni wrote: > > grep and CVS agree that this is a switch(4) left-over. > > > > OK? > > This is exported to userland isn't it? I seariously hope not. All those caddr_t are kernel pointers. In

bgpd/bgpctl report number of pending updates/withdraws

2022-08-29 Thread Claudio Jeker
The RDE has a queue of pending updates and withdraws. Those are already counted but not shown. On big setups it may be helpful to know about the queue progress. -- :wq Claudio Index: bgpctl/output.c === RCS file:

bgpd switch communities to RB tree

2022-08-29 Thread Claudio Jeker
The hash table for communities is not great. Instead of implementing dynamic hash resize use a RB tree. Also drop the hash calculation and just use memcmp() for now. My non scientific test seems to indicate that the overhead of SipHash is about the same as the memcmp(). -- :wq Claudio Index:

bgpd speedup diff

2022-08-29 Thread Claudio Jeker
On large bgpd instances the hash tables used for rde_aspath, aspath and communities get overloaded to a point that aspath_get() consumes a large amount of CPU time. This diff improves the situation by a) using a RB tree for rde_aspath and communities and b) dropping the hash table for aspath all

Re: bgpd/bgpctl report number of pending updates/withdraws

2022-08-29 Thread Claudio Jeker
On Mon, Aug 29, 2022 at 04:47:51PM +0200, Theo Buehler wrote: > On Mon, Aug 29, 2022 at 04:41:56PM +0200, Claudio Jeker wrote: > > The RDE has a queue of pending updates and withdraws. Those are already > > counted but not shown. On big setups it may be helpful to know about the >

bgpd switch rde_aspath to RB tree

2022-08-29 Thread Claudio Jeker
This is the 2nd RB tree transformation instead of the hash table. The rde.c part will probably conflict with the change for communities. -- :wq Claudio ? obj Index: rde.c === RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v retrieving

Re: rpki-client: use valid_uri() in load_skiplist()

2022-08-26 Thread Claudio Jeker
On Fri, Aug 26, 2022 at 01:48:55PM +0200, Theo Buehler wrote: > It occurred to me right after committing the previous change that it is > doing the same thing as valid_uri(). Calling it is simpler and the > additional "/." check won't hurt. This is indeed OK. What worries me a bit is that the

Re: bgpd switch communities to RB tree

2022-08-29 Thread Claudio Jeker
On Mon, Aug 29, 2022 at 05:36:49PM +0200, Theo Buehler wrote: > On Mon, Aug 29, 2022 at 05:05:59PM +0200, Claudio Jeker wrote: > > The hash table for communities is not great. Instead of implementing > > dynamic hash resize use a RB tree. Also drop the hash calculation and >

Re: bgpd switch communities to RB tree

2022-08-29 Thread Claudio Jeker
On Mon, Aug 29, 2022 at 05:45:58PM +0200, Claudio Jeker wrote: > On Mon, Aug 29, 2022 at 05:36:49PM +0200, Theo Buehler wrote: > > On Mon, Aug 29, 2022 at 05:05:59PM +0200, Claudio Jeker wrote: > > > The hash table for communities is not great. Instead of implementing > > &

Re: bgpd remove aspath cache for more speed

2022-08-29 Thread Claudio Jeker
On Mon, Aug 29, 2022 at 07:44:33PM +0200, Theo Buehler wrote: > On Mon, Aug 29, 2022 at 07:28:58PM +0200, Claudio Jeker wrote: > > This diff removes the aspath cache. I tried replacing it with an RB tree > > but it still consumes a lot of CPU cycles for little gain. So instea

bgpd remove aspath cache for more speed

2022-08-29 Thread Claudio Jeker
This diff removes the aspath cache. I tried replacing it with an RB tree but it still consumes a lot of CPU cycles for little gain. So instead just copy the aspath for all rde_aspath structs. It will use more memory but in most cases the amount is in the 20-30% order. This also includes a lot of

Re: bgpd switch rde_aspath to RB tree

2022-08-29 Thread Claudio Jeker
On Mon, Aug 29, 2022 at 05:49:06PM +0200, Theo Buehler wrote: > On Mon, Aug 29, 2022 at 05:28:50PM +0200, Claudio Jeker wrote: > > This is the 2nd RB tree transformation instead of the hash table. > > > > The rde.c part will probably conflict with the change for communi

Re: udp pcb mutex

2022-08-29 Thread Claudio Jeker
On Mon, Aug 29, 2022 at 07:34:14PM +0200, Alexander Bluhm wrote: > Hi, > > The diff below is needed to protect the receive socket buffer in > UDP input with per PCB mutex. > > With that, parallel UDP input and soreceive can be activated. There > are still issues with socket splicing and maybe

Re: diff: b64decode(1) for long line

2022-08-30 Thread Claudio Jeker
On Tue, Aug 30, 2022 at 11:18:01AM +0200, YASUOKA Masahiko wrote: > b64decode(8) fails if a long line is given. > > % wc test > 1 11370 test > % > % ./b64decode -r test > /dev/null > b64decode: test: /dev/stdout: error decoding base64 input stream > % > > uudecode.c > >

rpki-client rrdp cleanup race

2022-08-30 Thread Claudio Jeker
Once rpki-client finished validating everything rpki moves valid files into the validated cache. For RRDP invalid files currently in the valid cache are moved back into the corresponding .rrdp directory. The problem here is that in the .rrdp a same file exists we override the newer invalid file

bgpd more kroute cleanup

2022-08-18 Thread Claudio Jeker
It makes no sense to pass the fd to send_rtmsg() as an argument. The code just passes the fd from the global kr_state. It also makes the code less portable because for linux an mnl handle needs to be passed. By dropping this the code becomes simpler. -- :wq Claudio Index: kroute.c

bgpd, uninitalised check in kroute_insert()

2022-08-18 Thread Claudio Jeker
Noticed while compling with gcc. In kroute_insert() the check for possible multipath routes is: if (krm == NULL) kr_redistribute(IMSG_NETWORK_ADD, kt, kf); The problem is krm is only set in the IPv4 path but not in the IPv6 one. The diff below fixes this by using a new

bgpd: only run one roa softreconfig process at a time

2022-08-30 Thread Claudio Jeker
Currently if bgpd takes a long time to re-evaluate all prefixes because of a ROA change a second update can come in before the first is processed. This is not good. So add a barrier to only run one rde_roa_softreconfig dump at a time. If a dump is pending while a new roa set is received ignore

Re: add sendmmsg and recvmmsg systemcalls

2022-08-30 Thread Claudio Jeker
On Tue, Aug 30, 2022 at 09:51:46PM +0100, Stuart Henderson wrote: > btw a few ports will likely pick this up: > > paths/devel/glib2.log:Checking for function "recvmmsg" : NO > paths/net/tinc.log:checking for recvmmsg... no > paths/net/knot.log:checking for recvmmsg... no > paths/net/knot.log:

bgpd switch rde_peer to RB tree

2022-09-01 Thread Claudio Jeker
Convert the rde_peer hash table to an RB tree. This is a bit more complex because rde_peer list is used in a lot of places. As a bonus use peer_foreach in mrt.c to write the table v2 peer header (this needs a special callback struct because two values need to be passed to the callback). The rest

bgpd cleanup hash leftovers

2022-09-01 Thread Claudio Jeker
bgpd no longer needs siphash.h and also remove a hash member and a prototype which are now unused. -- :wq Claudio Index: rde.h === RCS file: /cvs/src/usr.sbin/bgpd/rde.h,v retrieving revision 1.268 diff -u -p -r1.268 rde.h ---

bgpd change attr cache to use RB tree

2022-08-31 Thread Claudio Jeker
Like all other hash tables use an RB tree instead. Again the calculation of the hash can be skipped because the compare function is probably fast enough. -- :wq Claudio ? ktrace.out ? obj Index: rde.c === RCS file:

Re: Rename global ifnet TAILQ

2022-08-31 Thread Claudio Jeker
On Wed, Aug 31, 2022 at 08:37:06AM +, Klemens Nanni wrote: > On Fri, Aug 26, 2022 at 06:14:44PM +, Klemens Nanni wrote: > > Naming the list like the struct itself makes for awful grepping. > > Distinguish the list name; no functional change. > > > > Builds/runs fine on and64 and sparc64.

rpki-client refactor rsync process

2022-09-02 Thread Claudio Jeker
The rsync process implements a limit by stopping to read commands from its stdin once too many processes are run. This is all nice and fine but it does not allow to send a abort request to the process reliably. This diff refactors the rsync process and introduces a state queue which can have more

rpki-client stop all repo fetching a bit before the timeout

2022-09-02 Thread Claudio Jeker
Lets try to finish work by stopping all syncs and fall back to what we have in cache after 7/8 of the timeout (timeout - 1/2 repo_timeout). This way we still have 1/8 of time to finish the calculation and produce output. Tested this diff by setting the deadline to fire after 60sec. -- :wq

rpki-client mini refactor

2022-09-03 Thread Claudio Jeker
Instead of passing the repo to queue_from_mft() do the lookup in the function. -- :wq Claudio Index: main.c === RCS file: /cvs/src/usr.sbin/rpki-client/main.c,v retrieving revision 1.218 diff -u -p -r1.218 main.c --- main.c 2

Re: rpki-client stop all repo fetching a bit before the timeout

2022-09-02 Thread Claudio Jeker
On Fri, Sep 02, 2022 at 09:50:06PM +, Job Snijders wrote: > Hi Claudio, > > This looks mostly OK, just a few nit: > > On Fri, Sep 02, 2022 at 10:02:33PM +0200, Claudio Jeker wrote: > > @@ -1223,8 +1224,26 @@ repo_check_timeout(int timeout) > > { > > st

rpki-client include "parent" repo id in certs

2022-09-03 Thread Claudio Jeker
This diff adds the parentid to struct cert. The parentid is the id of the repository the cert lives in. This information will be used to track the parent repository in the repositories list/tree. The naming is confusing and I'm happy for better suggestions. -- :wq Claudio Index: cert.c

rpki-client http client make header parser more RFC compliant

2022-09-07 Thread Claudio Jeker
The HTTP RFC defines a header as: message-header = field-name ":" [ field-value ] field-name = token field-value= *( field-content | LWS ) The field-content does not include any leading or trailing LWS: linear white space occurring before the first non-whitespace

ftp follow RFC 9112 more closely

2022-09-07 Thread Claudio Jeker
When parsing header lines ftp (as does rpki-client) is a bit too strict in what it accepts. In short the field values can have leading and trailing whitespace which needs to be ignored. See RFC9112 Section 5. ftp only allows exactly a single leading space which is not in spec. This is a very

bgpctl remove hash stats

2022-08-31 Thread Claudio Jeker
bgpctl show rib mem no longer shows any hash statistics so remove that code from bgpctl as a first step. Also since show rib mem is a single message adjust the code to be done after the message is received. OK? -- :wq Claudio Index: bgpctl.c

bgpd kill hash statistic imsg

2022-08-31 Thread Claudio Jeker
IMSG_CTL_SHOW_RIB_HASH is no longer used, GC it. Also kill some global hash sizes which are not used anymore. -- :wq Claudio Index: bgpd.h === RCS file: /cvs/src/usr.sbin/bgpd/bgpd.h,v retrieving revision 1.451 diff -u -p -r1.451

Re: diff: b64decode(1) for long line

2022-08-30 Thread Claudio Jeker
On Tue, Aug 30, 2022 at 02:19:29PM +0200, Theo Buehler wrote: > > Ah, I showed the diff separated from first one. > > I'm sorry for confusing. The following diff is combined. > > Oh, I see. > > ok tb (feel free to land the diffs separately if you prefer). Also OK claudio -- :wq Claudio

bgpd move nexthops to RB tree

2022-08-30 Thread Claudio Jeker
I'm on a mission to remove the hash tables :) This one is for struct nexthop. Hopefully it makes nexthop_get a bit better. -- :wq Claudio Index: rde.c === RCS file: /cvs/src/usr.sbin/bgpd/rde.c,v retrieving revision 1.569 diff -u

bgpd fix peer signaling bug for busy systems

2022-08-26 Thread Claudio Jeker
Noticed on a route collector with >100 full feeds and well 80Mio prefixes. On startup the RDE slurps in a lot of messages and then slowly processes them. Those are mostly IMSG_UDPATE but the current code also queues IMSG_SESSION_DOWN, IMSG_SESSION_UP and the graceful restart imsgs. It does not

Re: rpki-client: two skiplist tweaks

2022-08-26 Thread Claudio Jeker
On Fri, Aug 26, 2022 at 09:57:19AM +0200, Theo Buehler wrote: > First, if there's an issue opening the default skip list file other than > its absence (most likely bad permissions), we should not silently ignore > it. Also, let's display the error, so use err(). > > Second, linelen, the return

Re: all architectures: put clockframe definition in frame.h?

2022-08-19 Thread Claudio Jeker
On Thu, Aug 18, 2022 at 10:32:36PM -0500, Scott Cheloha wrote: > Hi, > > clockframe is sometimes defined in cpu.h, sometimes in frame.h, and > sometimes defined once each in both header files. > > Can we put the clockframe definitions in frame.h? Always? It is, at > least ostensibly, a

bgpd fix nexthop lookup for connected networks

2022-08-19 Thread Claudio Jeker
When implementing knexthop_true_nexthop() to do the lookup from BGP nexthop to the true nexthop used by the FIB I forgot to handle connected networks properly. For connected networks and connected nexthops the BGP exit nexthop is equal to the true nexthop used by the FIB since the nexthop is

bgpctl remove extra space in ovs output

2022-09-28 Thread Claudio Jeker
There is an extra space in the output of ext communities of type OVS. It seems when fmt_ext_community was introduced the extra space was not removed from those special ext-communities. This was noticed by job@ -- :wq Claudio Index: bgpctl.c

initialize peer last_updown when peer is created

2022-10-18 Thread Claudio Jeker
Currently the last_updown stat is not initalized and 0 when a peer is initially added. If the peer is passive or is unable to establish a connection last_updown remains 0 which is not ideal. Initalize the last_updown timestamp in init_peer() which is called when a new peer is added or cloned.

fix use after free in proxy_parse_uri()

2022-10-18 Thread Claudio Jeker
With rev 1.65 proxy_parse_uri() can assign a pointer to proxyport that is part of fullhost and so points to freed memory (once that function returns). The fix is to copy the port as well. This should be a fix for https://github.com/rpki-client/rpki-client-portable/issues/74 -- :wq Claudio

bgpctl show metric up/down time

2022-10-18 Thread Claudio Jeker
As mentioned I think having metric values that depend on session state is not ideal. Introduce 'bgpd_peer_last_change_seconds' which is the last time the session up/down state changed. It does not track every state change only when a session goes in and out of ESTABLISHED state. This replaces

Re: bgpctl openmetric/prometheus output

2022-10-17 Thread Claudio Jeker
On Mon, Oct 17, 2022 at 12:39:44PM +0200, Denis Fondras wrote: > Le Mon, Oct 17, 2022 at 11:49:31AM +0200, Claudio Jeker a écrit : > > On Wed, Oct 12, 2022 at 12:12:25PM +0200, Theo Buehler wrote: > > > On Fri, Oct 07, 2022 at 12:37:10PM +0200, Claudio Jeker wrote:

Re: bgpctl openmetric/prometheus output

2022-10-17 Thread Claudio Jeker
On Wed, Oct 12, 2022 at 12:12:25PM +0200, Theo Buehler wrote: > On Fri, Oct 07, 2022 at 12:37:10PM +0200, Claudio Jeker wrote: > > This diff adds `bgpctl show metric` which is a command that dumps some > > stats out in openmetric format. This format can be ingested by e.g. > >

bgplgd add metrics api endpoint for open metrics

2022-10-17 Thread Claudio Jeker
This diff adds the required bits to export the bgpctl show metrics output as openmetrics. For this /metrics needs to answer with a different content-type. Apart from that adding this is trivial. -- :wq Claudio Index: bgplgd.c ===

<    14   15   16   17   18   19   20   21   22   23   >