This has been changed in Ubuntu 12.04 - there is no longer an automatic switch to HDMI. You can see http://voices.canonical.com/david.henningsson/2012/04/14/audio-over-hdmi-and-displayport-in-ubuntu-12-04/ for some background information and links to the discussion about this topic.
** Changed in: alsa-driver (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1000881 Title: Automatic switch of audio to HDMI output not wanted. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/alsa-driver/+bug/1000881/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs