I've the same exact problem, i have alternately installed xfce4.4 from the xfce.org installer, and don't have these problems.
Both Xfce-desktop and xfce-menu-plugin, progressively use more memory. Eventually rendering xfce practically unusable. In my experience each has reached around 250/300 Mb of memory, after around 48 hours. -- Xfce4.4 using too much memory; possible memory leaks https://launchpad.net/bugs/87925 -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs