I have been told that I can replace the glass mirror without removing the mirror housing.
Absolutely. The job is either trivial or merely easy, depending on whether or not you have heated mirrors. (I think.) Some glass, the heated kind I thought, is retained by a circular metal spring clip you release by folding the mirror head back against the car and poking around through the open hinge. The non-heated kind just pries out. If you try to pry out one that's clip-retained you'll break the glass. -- Jim