I agree...and assummed he knew this too. He wanted users to end up at a
different physical server and without a second failover solution that was
the only thing I could think of to try and explain how users ended up at his
servers. Once you hit squid, you are there. If you look for squid and it's
failed, you are stuck, it can't just pass you through anyways.
Exactly :-)
I think your mentioned "second failover" would be the solution. Two squid boxes either with a load balancer(s) or a cluster software should do the trick, but the ultimate goal would be to eliminate all single point of failures. Internet connection, firewall, reverse proxy, webserver, databases.... On the other hand HA systems are more complex and may fail to switch over if the time has come.
A simpler approach would be to make every box as reliable as possible.
Decent hardware, a hardened setup, a good monitoring, a standby server and an operator in stand-by (hope I found the right word for this :-).
There is no use to build an HA system on crappy hardware...
As for squid I think it can run ages without any problem, I have never seen squid itself crashing on its own, it was always my fault - e.g. misconfiguration.
Regards, Hendrik Voigtländer