I found out the reason this happens a few of days back. Just to let you
know..
It seems it has partly to do with the way we handle reboots on our setup.
When we take down one of our replica servers (for testing/maintenance), to
ensure that the bricks are unmounted correctly, we kill off the
CC gluster-users.
No, there aren't any firewall rules in our server. As I write in one of my
earlier emails, if I kill the mount client, and remount the volume, then
the problem disappears. That is to say, this causes the client to refresh
remote port data and from there everything's fine.