Re: net.add_addr_allfibs=1 behaviour deprecation
On 18/08/2020 19:27, Alexander V. Chernikov wrote: 18.08.2020, 09:17, "Grzegorz Junka" : On 18/08/2020 07:54, Julian Elischer wrote: No objections has been received. Next steps: * Switch net.add_addr_allfibs to 0 ( https://reviews.freebsd.org/D26076 ) * Provide an ability to use nexthops from different fibs * Remove net.add_addr_allfibs Timeline: Aug 1: summarising feedback and the usecases, decision on proceeding further Aug 20 (tentative): patches for supported usecases Sep 15 (tentative): net.add_addr_allfibs removal. [1]: [base Contents of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup) [2]: [base Diff of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) /Alexander Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing installations, which goes against the least surprise principle so many times advocated on FreeBSD lists. This is just one example: https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/ Now, changing the default again might again break existing installations, which shouldn't be a reason for not doing it, but might be a reason to better communicate it this time around. I plan to communicate it the following way: 1) this thread 2) GONE_IN13 in the sysctl (which will print console message when set to 1) 3) Release notes Do you think there are other communication channels one should try to use? When I was looking for information about why the routing doesn't work as expected I didn't know about this settings but couldn't find anything relevant in the Handbook, so would really love to see this somehow mentioned and explained how to use it in the Handbook if possible. Thanks GrzegorzJ ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: net.add_addr_allfibs=1 behaviour deprecation
18.08.2020, 09:17, "Grzegorz Junka" : > On 18/08/2020 07:54, Julian Elischer wrote: >> The reason for the two behaviours is that there are two ways that the >> previous behaviour of "add addresses to the only FIB" could be >> interpreted and extended once multiple fibs became available. The >> single fib case could be interpreted as either of: >> >> "Add to All N fibs where N == 1" or "add to the first (of 1) >> fibs". >> I decided to do both :-) >> >> At Ironport where I wrote it we had a scenario where I didn't want to >> have wrong entries in all the fibs when a new interface was brought >> up. Even for a moment. An other scenarios where for example a tunnel >> uses fib 1 but the rest of the system uses fib0 (which points through >> the tunnel) The addition of new routes into the tunnel's route when a >> new virtual interface is brought up pointing through the tunnel to the >> same address, leads in the tunnel immediately redirecting packets >> through itself which ends in tears. SO the obvious thing to do was >> to make it possible to only add the entry in the fib that was the >> default fib or in the case of Ironport, the fib that was the default >> fib of the process adding the interface. >> >> If you had to make a choice I think the '0' choice is the way to go. >> All other fibs need to be populated deliberately.. >> >> On 8/15/20 4:24 AM, Alexander V. Chernikov wrote: >>> 18.07.2020, 14:22, "Alexander V. Chernikov" : Dear FreeBSD users, I would like to make net.add_addr_allfibs=0 as the default system behaviour and remove net.add_addr_allfibs. To do so, I would like to collect use cases with net.add_addr_allfibs=1 and multiple fibs, to ensure they can still be supported after removal. Background: Multi-fib support was added in r17 [1], 12 years ago. Addition of interface addresses to all fibs was a feature from day 1. The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 years ago. Problem: The goal of the fib support is to provide multiple independent routing tables, isolated from each other. `net.add_addr_allfibs` default tries to shift gears in the opposite direction, unconditionally inserting all addresses to all of the fibs. It complicates the logic, kernel code and makes control plane performance decrease with the number of fibs. It make impossible to use the same prefixes in multiple fibs, which may be desired given shortage of IPv4 address space. I do understand that there are some cases where such behaviour is desired. For example, it can be used to achieve VRF route leaking or binding on address from different fibs. I would like to collect such cases to consider supporting them in a different way. The goal is to make net.add_addr_allfibs=0 default behaviour and remove net.add_addr_allfibs. It will simplify kernel fib-related code and allow bringing more fib-related features. It will also improve fib scaling. >>> No objections has been received. >>> Next steps: >>> * Switch net.add_addr_allfibs to 0 ( >>> https://reviews.freebsd.org/D26076 ) >>> * Provide an ability to use nexthops from different fibs >>> * Remove net.add_addr_allfibs Timeline: Aug 1: summarising feedback and the usecases, decision on proceeding further Aug 20 (tentative): patches for supported usecases Sep 15 (tentative): net.add_addr_allfibs removal. [1]: [base Contents of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup) [2]: [base Diff of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) /Alexander > > Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing > installations, which goes against the least surprise principle so many > times advocated on FreeBSD lists. > > This is just one example: > https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/ > > Now, changing the default again might again break existing > installations, which shouldn't be a reason for not doing it, but might > be a reason to better communicate it this time around. I plan to communicate it the following way: 1) this thread 2) GONE_IN13 in the sysctl (which will print console message when set to 1) 3) Release notes Do you think there are other communication channels one should try to use? > > GrzegorzJ > > ___ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net
Re: net.add_addr_allfibs=1 behaviour deprecation
___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: net.add_addr_allfibs=1 behaviour deprecation
On 18/08/2020 07:54, Julian Elischer wrote: The reason for the two behaviours is that there are two ways that the previous behaviour of "add addresses to the only FIB" could be interpreted and extended once multiple fibs became available. The single fib case could be interpreted as either of: "Add to All N fibs where N == 1" or "add to the first (of 1) fibs". I decided to do both :-) At Ironport where I wrote it we had a scenario where I didn't want to have wrong entries in all the fibs when a new interface was brought up. Even for a moment. An other scenarios where for example a tunnel uses fib 1 but the rest of the system uses fib0 (which points through the tunnel) The addition of new routes into the tunnel's route when a new virtual interface is brought up pointing through the tunnel to the same address, leads in the tunnel immediately redirecting packets through itself which ends in tears. SO the obvious thing to do was to make it possible to only add the entry in the fib that was the default fib or in the case of Ironport, the fib that was the default fib of the process adding the interface. If you had to make a choice I think the '0' choice is the way to go. All other fibs need to be populated deliberately.. On 8/15/20 4:24 AM, Alexander V. Chernikov wrote: 18.07.2020, 14:22, "Alexander V. Chernikov" : Dear FreeBSD users, I would like to make net.add_addr_allfibs=0 as the default system behaviour and remove net.add_addr_allfibs. To do so, I would like to collect use cases with net.add_addr_allfibs=1 and multiple fibs, to ensure they can still be supported after removal. Background: Multi-fib support was added in r17 [1], 12 years ago. Addition of interface addresses to all fibs was a feature from day 1. The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 years ago. Problem: The goal of the fib support is to provide multiple independent routing tables, isolated from each other. `net.add_addr_allfibs` default tries to shift gears in the opposite direction, unconditionally inserting all addresses to all of the fibs. It complicates the logic, kernel code and makes control plane performance decrease with the number of fibs. It make impossible to use the same prefixes in multiple fibs, which may be desired given shortage of IPv4 address space. I do understand that there are some cases where such behaviour is desired. For example, it can be used to achieve VRF route leaking or binding on address from different fibs. I would like to collect such cases to consider supporting them in a different way. The goal is to make net.add_addr_allfibs=0 default behaviour and remove net.add_addr_allfibs. It will simplify kernel fib-related code and allow bringing more fib-related features. It will also improve fib scaling. No objections has been received. Next steps: * Switch net.add_addr_allfibs to 0 ( https://reviews.freebsd.org/D26076 ) * Provide an ability to use nexthops from different fibs * Remove net.add_addr_allfibs Timeline: Aug 1: summarising feedback and the usecases, decision on proceeding further Aug 20 (tentative): patches for supported usecases Sep 15 (tentative): net.add_addr_allfibs removal. [1]: [base Contents of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup) [2]: [base Diff of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) /Alexander Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing installations, which goes against the least surprise principle so many times advocated on FreeBSD lists. This is just one example: https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/ Now, changing the default again might again break existing installations, which shouldn't be a reason for not doing it, but might be a reason to better communicate it this time around. GrzegorzJ ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: net.add_addr_allfibs=1 behaviour deprecation
The reason for the two behaviours is that there are two ways that the previous behaviour of "add addresses to the only FIB" could be interpreted and extended once multiple fibs became available. The single fib case could be interpreted as either of: "Add to All N fibs where N == 1" or "add to the first (of 1) fibs". I decided to do both :-) At Ironport where I wrote it we had a scenario where I didn't want to have wrong entries in all the fibs when a new interface was brought up. Even for a moment. An other scenarios where for example a tunnel uses fib 1 but the rest of the system uses fib0 (which points through the tunnel) The addition of new routes into the tunnel's route when a new virtual interface is brought up pointing through the tunnel to the same address, leads in the tunnel immediately redirecting packets through itself which ends in tears. SO the obvious thing to do was to make it possible to only add the entry in the fib that was the default fib or in the case of Ironport, the fib that was the default fib of the process adding the interface. If you had to make a choice I think the '0' choice is the way to go. All other fibs need to be populated deliberately.. On 8/15/20 4:24 AM, Alexander V. Chernikov wrote: 18.07.2020, 14:22, "Alexander V. Chernikov" : Dear FreeBSD users, I would like to make net.add_addr_allfibs=0 as the default system behaviour and remove net.add_addr_allfibs. To do so, I would like to collect use cases with net.add_addr_allfibs=1 and multiple fibs, to ensure they can still be supported after removal. Background: Multi-fib support was added in r17 [1], 12 years ago. Addition of interface addresses to all fibs was a feature from day 1. The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 years ago. Problem: The goal of the fib support is to provide multiple independent routing tables, isolated from each other. `net.add_addr_allfibs` default tries to shift gears in the opposite direction, unconditionally inserting all addresses to all of the fibs. It complicates the logic, kernel code and makes control plane performance decrease with the number of fibs. It make impossible to use the same prefixes in multiple fibs, which may be desired given shortage of IPv4 address space. I do understand that there are some cases where such behaviour is desired. For example, it can be used to achieve VRF route leaking or binding on address from different fibs. I would like to collect such cases to consider supporting them in a different way. The goal is to make net.add_addr_allfibs=0 default behaviour and remove net.add_addr_allfibs. It will simplify kernel fib-related code and allow bringing more fib-related features. It will also improve fib scaling. No objections has been received. Next steps: * Switch net.add_addr_allfibs to 0 ( https://reviews.freebsd.org/D26076 ) * Provide an ability to use nexthops from different fibs * Remove net.add_addr_allfibs Timeline: Aug 1: summarising feedback and the usecases, decision on proceeding further Aug 20 (tentative): patches for supported usecases Sep 15 (tentative): net.add_addr_allfibs removal. [1]: [base Contents of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup) [2]: [base Diff of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) /Alexander ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org" ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: net.add_addr_allfibs=1 behaviour deprecation
On Sat, Aug 15, 2020 at 5:25 AM Alexander V. Chernikov wrote: > 18.07.2020, 14:22, "Alexander V. Chernikov" : > > Dear FreeBSD users, > > > > I would like to make net.add_addr_allfibs=0 as the default system > behaviour and remove net.add_addr_allfibs. > > To do so, I would like to collect use cases with net.add_addr_allfibs=1 > and multiple fibs, to ensure they can still be supported after removal. > > > > Background: > > > > Multi-fib support was added in r17 [1], 12 years ago. Addition of > interface addresses to all fibs was a feature from day 1. > > The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 years > ago. > > > > Problem: > > The goal of the fib support is to provide multiple independent routing > tables, isolated from each other. > > `net.add_addr_allfibs` default tries to shift gears in the opposite > direction, unconditionally inserting all addresses to all of the fibs. > > > > It complicates the logic, kernel code and makes control plane > performance decrease with the number of fibs. > > It make impossible to use the same prefixes in multiple fibs, which may > be desired given shortage of IPv4 address space. > > > > I do understand that there are some cases where such behaviour is > desired. > > For example, it can be used to achieve VRF route leaking or binding on > address from different fibs. > > I would like to collect such cases to consider supporting them in a > different way. > > > > The goal is to make net.add_addr_allfibs=0 default behaviour and remove > net.add_addr_allfibs. > > It will simplify kernel fib-related code and allow bringing more > fib-related features. It will also improve fib scaling. > No objections has been received. > Next steps: > * Switch net.add_addr_allfibs to 0 ( https://reviews.freebsd.org/D26076 ) > * Provide an ability to use nexthops from different fibs > * Remove net.add_addr_allfibs > > Timeline: > > Aug 1: summarising feedback and the usecases, decision on proceeding > further > > Aug 20 (tentative): patches for supported usecases > > Sep 15 (tentative): net.add_addr_allfibs removal. > > > > [1]: [base Contents of /head/sys/net/route.c]( > https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup > ) > > [2]: [base Diff of /head/sys/net/route.c]( > https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) > > > > /Alexander > I just want to say that I completely agree with this proposal. -Alan ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
Re: net.add_addr_allfibs=1 behaviour deprecation
18.07.2020, 14:22, "Alexander V. Chernikov" : > Dear FreeBSD users, > > I would like to make net.add_addr_allfibs=0 as the default system behaviour > and remove net.add_addr_allfibs. > To do so, I would like to collect use cases with net.add_addr_allfibs=1 and > multiple fibs, to ensure they can still be supported after removal. > > Background: > > Multi-fib support was added in r17 [1], 12 years ago. Addition of > interface addresses to all fibs was a feature from day 1. > The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 years ago. > > Problem: > The goal of the fib support is to provide multiple independent routing > tables, isolated from each other. > `net.add_addr_allfibs` default tries to shift gears in the opposite > direction, unconditionally inserting all addresses to all of the fibs. > > It complicates the logic, kernel code and makes control plane performance > decrease with the number of fibs. > It make impossible to use the same prefixes in multiple fibs, which may be > desired given shortage of IPv4 address space. > > I do understand that there are some cases where such behaviour is desired. > For example, it can be used to achieve VRF route leaking or binding on > address from different fibs. > I would like to collect such cases to consider supporting them in a different > way. > > The goal is to make net.add_addr_allfibs=0 default behaviour and remove > net.add_addr_allfibs. > It will simplify kernel fib-related code and allow bringing more fib-related > features. It will also improve fib scaling. No objections has been received. Next steps: * Switch net.add_addr_allfibs to 0 ( https://reviews.freebsd.org/D26076 ) * Provide an ability to use nexthops from different fibs * Remove net.add_addr_allfibs > Timeline: > Aug 1: summarising feedback and the usecases, decision on proceeding further > Aug 20 (tentative): patches for supported usecases > Sep 15 (tentative): net.add_addr_allfibs removal. > > [1]: [base Contents of > /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup) > [2]: [base Diff of > /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) > > /Alexander ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
net.add_addr_allfibs=1 behaviour deprecation
Dear FreeBSD users, I would like to make net.add_addr_allfibs=0 as the default system behaviour and remove net.add_addr_allfibs. To do so, I would like to collect use cases with net.add_addr_allfibs=1 and multiple fibs, to ensure they can still be supported after removal. Background: Multi-fib support was added in r17 [1], 12 years ago. Addition of interface addresses to all fibs was a feature from day 1. The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 years ago. Problem: The goal of the fib support is to provide multiple independent routing tables, isolated from each other. `net.add_addr_allfibs` default tries to shift gears in the opposite direction, unconditionally inserting all addresses to all of the fibs. It complicates the logic, kernel code and makes control plane performance decrease with the number of fibs. It make impossible to use the same prefixes in multiple fibs, which may be desired given shortage of IPv4 address space. I do understand that there are some cases where such behaviour is desired. For example, it can be used to achieve VRF route leaking or binding on address from different fibs. I would like to collect such cases to consider supporting them in a different way. The goal is to make net.add_addr_allfibs=0 default behaviour and remove net.add_addr_allfibs. It will simplify kernel fib-related code and allow bringing more fib-related features. It will also improve fib scaling. Timeline: Aug 1: summarising feedback and the usecases, decision on proceeding further Aug 20 (tentative): patches for supported usecases Sep 15 (tentative): net.add_addr_allfibs removal. [1]: [base Contents of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=17=markup) [2]: [base Diff of /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839=180840;) /Alexander ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
