Quoting Andrew Deason <adea...@sinenomine.net>:
Is the "internal" address only routable from that site?
No. Although each of the three sites will be connected to a small internal network with private IP addresses, the broadband modems will be in bridged mode and the server will also have two public IP addresses. Each server will also act as a router between the internal network and the external links (but not between the latter). I figure that it will be best if each OpenAFS server is only known by its public IP address(es), even to the clients on its internal network. This way, each OpenAFS client at each location in the cell will always contact the three OpenAFS servers using the same set of public IP addresses.
If you're just taking the "real" sites and adding additional IPs, though, I think the clients should be fine. Aside from the aforementioned sync site determination, it's effectively just a list of sites to try to contact.
Okay, so I could give all of the clients a list of six IP addresses for the three servers. I plan to use DNS with AFSDB RRs for this, although that doesn't provide a method for giving priority to the faster links. However, I recently noticed RFC5864 (good work, Russ!): what's the minimum OpenAFS version necessary for anyone wanting to use SRV RRs for OpenAFS instead?
dbservers are different, though; the specification in CellServDB affects the voting algorithm, and is not as simple as a list of sites to try to reach. I don't think I'd recommend putting multiple IPs per site in there, but I don't really know what happens if you do that. Each site is aware of the local addresses of the other sites, and I'm not sure what a site does if it sees that another site claims ownership of more than one CellServDB server entry... But I don't imagine it ending well.
Yeah, I suspected that would be a problem... despite the fact that the names following the "#" behind the IP addresses would be the same. So, I'll just use one IP address per server in the CellServDB files of the DB/file server machines and let the routing system take care of any communications problems.
As to whether this is advisable in general... it's not immediately clear to me what failure case you're trying to cover here. If you have three geographically-diverse sites, losing access to one of those db sites will not significantly affect accessing AFS beyond an initial delay when the site goes away. If you lose two sites, then you lose the ability to make vldb or ptdb changes, but you can still access data (assuming the relevant fileservers are still accessible).
The redundant links will significantly reduce the risk for each site that, if it temporarily loses its connection to the Internet, the local OpenAFS file server also becomes read-only until the link is restored. But, besides that, the business already grinds to a halt when Internet access is lost due to their dependence on a proprietary online database management solution. So, the redundant links will also address an existing problem.
... it doesn't seem worth the effort of running with such a non-typical configuration. While it may be possible to make it work, providing the redundancy at a lower level seems easier.
Indeed. It was always the plan to implement that low-level redundancy anyway.
What would seem to have more immediate benefit is to have your fileservers register (at least) two IP addresses in the vldb; one for each redundant link. That way, if one link to the site containing that fileserver goes down, clients should still be able to access the data on that fileserver.
Would it then not be necessary to have the database and file servers installed on separate hosts? Otherwise they would share the same CellServDB files (even with a single physical machines, that's possible using virtual hosts, although I would not be able to afford two public IPv4 addresses for both hosts). Or is that not what you mean?
Of course, this should already happen automatically if the two IPs for the fileserver are addresses known by its local interfaces (that is, they show up in 'ifconfig' or whatever).
I plan to use the "iproute" package in Debian. Cheers, Jaap _______________________________________________ OpenAFS-info mailing list OpenAFS-info@openafs.org https://lists.openafs.org/mailman/listinfo/openafs-info
