Does anyone else find this behavior too strict?
Yes. Sounds like yet another case of forcing bitwise const on something that should only be logically const.
Please stop forcing bitwise constancy on everything. Not everything needs to work with immutable, it should be opt-in (which in this case could mean explicitly marking the invariant as const) for any type.