This happens when I remove emacs from the default profile `guix remove emacs` and install it via `guix home ...`. IOW the bug is not that $GUIX_PROFILE/share/emacs/site-lisp is missing in EMACSLOADPATH, but that "$HOME/.guix-profile/share/emacs/site-lisp" is in the load-path even if emacs is not installed in the default profile.
Cheers Bost