James, Thanks for your comments. See inline: > >> - IFF_NOACCEPT >> >> When a VRRP router becomes master, if its accept_mode is false, all >> the virtual IP addresses will be brought up but with the new >> IFF_NOACCEPT flag set by vrrpd. IP will mark all the local IREs >> associated with these IFF_NOACCEPT IP addresses to be "no_accept", >> and all the received unicast local packets will be dropped, with >> the exception of the Neighbor Solicitation packets and Neighbor >> Advertisement packets. This allows the Neighbor Unreachability >> Detection mechanism work as expected in the false accept_mode. > > This seems to describe IFF_NOACCEPT as a per-ipif flag. Is that the > right granularity? It seems odd to me that we could potentially > configure the system to have some addresses that respond and some that > do not. The point of "no accept" mode is to behave as a real VRRP > system. If you want that, I don't think you'd want it just some of > the time, would you? You are right. I can add a per-ill flag. But since I would also like this flag to be observable by the administrator, I will keep the IPIF_NOACCEPT flag so that ifconfig bge0:1 will be able to show this address is set to be "no_accept". > >> The IFF_NOACCEPT flag can only be set on a IP addresses if its >> IFF_VRRP flag is set. Further, the same SO_VRRP socket option must >> be set to change the IFF_NOACCEPT flag over a IP address. > > What's the rationale for requiring SO_VRRP? Nothing else knows about > IFF_NOACCEPT, so (unlike IFF_UP) it can't be set "accidentally" by > anyone else. (And you clearly can't *stop* appropriately privileged > applications from setting SO_VRRP on their own.) First, an application could potentially make a mistake by not getting ifflag before setting ifflag, which could potentially clear IFF_NOACCEPT. Second, this is also related to setting/clearing the IFF_UP flag over an VRRP VNIC: a non-accept VRRP router usually sets IFF_UP and IFF_NOACCEPT using the same socket when it becomes master, having SO_VRRP is to make the socket setting both flags to becomes a VRRP control socket session.
> A separate question on the IFF_UP limitations: I'm not sure why that's > being controlled by the VRRP daemon. Why not allow IFF_UP to be the > normal administrative control on the address, and just have the VRRP > daemon control whether the address is "live" on the network (i.e., > turning on and off ARP and ND for the address, as well as controlling > input and output)? > > This proposal looks workable, but it still looks a little more complex > to administer than I'd expected. You mean another flag to control the liveness of the network? We thought about that option, and eventually we choose today's approach is that there is ambiguity of the "IFF_UP" flag. It can be seens as an administrative flag, but some application also believes it shows the current state of the address: once it is UP, the address should be usable. > >> Since the existing PF_ROUTE event opcodes only report the changes >> of the set of "UP" IP addresses, and the vrrpd daemon needs to >> track all the virtual IP addresses configured over each VRRP router, >> regardless the IP address is brought up or not. Therefore, two >> new PF_ROUTE event opcodes will be introduced to report the changes >> of the IP addresses configuration, including the IP addresses that >> have not been brought up: >> >> - RTM_CHGADDR >> >> The RTM_CHGADDR event will be generated when a new IP address >> is newly configured (added or updated to). >> >> - RTM_FREEADDR >> >> The RTM_FREEADDR event will be generated when a IP address is >> removed from the configuration. > > These don't look like messages on other systems. Are there any > parallels with existing BSD extensions? I looked but didn't see things similar to it. BSD does provide a IF_ANNOUNCE message to indicate the plumbing/unplumbing of the interface but that cannot satisfy my needs. > > (I assume that "configure" means "plumb an ipif" here.) Right. > >> Note that both events will not report the unspecified (all-zero) >> IP addresses. > > I'm not sure what that means. Why would those need to be special? Or > is that just an implementation limitation? > > Aren't all ipifs constructed with zero address when first plumbed ... ? It is related to implementation. Usually, the first zero-address is the result of the SIOCLIFADDIF, but the IP address is set as the result of the SIOCSLIFADDR. We only generate the RTM_CHGADDR as a result of the second operation. Also, I can imagine that application could care about the plumbing event of the interface, but I assume that no application would care about the first zero-IP addresses, to me, that zero-address is a side effect of our implementation. > >> Since in some cases, the VRRP configuration is pretty simple and >> the administrator would prefer to have one single administrative >> tool to configure everything related to a VRRP router. Therefore, >> we give the option to an administrator to do that using only >> vrrpadm: the optional "-a" option can be used to specify a list >> of virtual IP addresses when creating a VRRP router, the "-P" >> option can be used to specify the primary IP address that >> is used to send VRRP advertisement packets. Further, if the "-f" >> option is specified, vrrpadm will create/plumb the VNIC if that >> has not been done: >> >> # vrrpadm create-router -V 12 -l bge1 -A inet -p 100 -o no_accept \ >> -a 11.1.1.1/24 -P 11.1.1.100/24 -f vrrp1 > > When the user does this, where is the information stored? Does > "create-router" write the address into /etc/hostname.*, and so on? The vrrp configuration stores in /etc/vrrp.conf, and we will make use of libipadm to do the ip configuration persistence. > >> - priv_net_rawaccess >> >> Required to hold the physical data-link (which owns the >> primary IP address) and the vnic (which owns the virtual >> IP addresses) open to prevent them from being deleted. > > Why is that necessary? To prevent the data-link from being deleted or renamed. Note that we don't require the underlying interface and VNIC to be plumbed when the router is enabled. Instead, the PF_ROUTE socket will track all the IP addresses configured over the underlying interface and VNIC and determine the primary IP address and virtual IP addresses. > >> If an administrator configures either IPv6 autoconfiguration or >> DHCP over a VRRP VNIC, since neither in.ndpd or dhcp client sets >> the SO_VRRP socket option, the attempt to bring up the >> auto-configured IP address will fail, which causes the failure >> of the autoconfiguration operation. > > Shouldn't these things be modified to check for IFF_VRRP and emit > reasonable messages and/or just avoid touching them? Sure. I will see what I can do. > > Relying on SO_VRRP and ioctl failure seems reasonable for protection > from third party applications, but I think our own applications should > probably do better. > >> After discussion with the team that potentially would consume >> this feature (the ILB team), > > Note that Quagga routing could consume this feature as well. (Perhaps > even sooner than ILB, as we've already got Quagga ...) > > > we found that this is an >> incomplete solution which does not simplify the management >> complexity. Ideally, the HA support configuration solution >> would be: >> configuring the following on the owner of the virtual IP >> addresses: - the virtual IP address(es) >> - the VRRP VRID >> - the IP addresses of the backup(s) >> - the ILB server groups, rules, hc, etc >> - some security configiguration (ssh keys) that allow >> automatic synchronization of things with the backups >> >> and all the other configurations and sync up between master and >> backup should be automatic. >> >> Clearly, the protected service support proposed by the old VRRP >> design does not help with the ideal world, therefore, we decide >> to remove the protected service support. > > That seems reasonable. Is there some way that SMF or Quagga could be > configured to signal VRRP when a routing daemon fails? VRRP also needs to know that which virtual addresses those services are using and find out the specific VRRP router. In theory, I think that can be done and can be a useful feature, but this will be out of the scope of the current VRRP/ILB project. > >> /usr/sbin/vrrpd Project Private VRRP daemon > > Nit: this probably belongs in /usr/lib instead. No user or > administrator should be starting VRRP on his own, should he? Sure. I can change that. Thanks - Cathy
