At Vimeo we have a custom tool since 2015 that monitors the membership of clusters of servers, templates out a config with servers assigned to backends, and manages reloading haproxy. We're looking into replacing this with something a bit more off-the-shelf, and one of the options is HAProxy's own DNS service discovery support.
We're also using URI-based load balancing with consistent hashing, and the stability of that mapping is important to us. Temporary disagreements while membership is changing are inevitable, but we want the portion of the hash space that a backend server sees to change as little as possible during its lifetime, and for multiple haproxies running the same config, against the same cluster, to converge on the same mapping. Our existing tool assigns a persistent ID to each server, which is mapped to an "id" option in the server line, which has worked quite well. >From what we've seen in testing so far, using "server-template" with DNS *doesn't* give us the behavior we want — the assignment of servers to slots seems inconsistent, maybe depending on some combination of the order of answers in the DNS packet or the order that new server appearances are observed by haproxy. Long story short: 1. Is my interpretation right? 2. Would you be open to a patch to change that? I'm thinking of something like setting puid from a hash of the SRV name or the A address, "open addressing" style, with who goes first in case of a collision determined by lexicographic order — but I'm quite open to guidance. Or should I just look somewhere other than the DNS service discovery? Thanks, Andrew Rodland (Please CC, I'm not on the list.)
