There's no need. I'll do the implementation with the prefix, and if you do it with a global hashtable within the same or better speed, my hat is off to you.
That is false dichotomy. What about storing the metadata at an address that is computable from from the object's address, while not being contiguous with the object allocated ? Is substracting a constant really the only option here ? (hint, it is not)