tinycore ships with lightweight Xvesa and a very simple packaging system. You don't have to strip anything off, but you'll have to add all clutter by yourself. The base system is about 10 megs, but you can add packages fast.
I would like to have more suckless folks sticking around, anybody can help. There are no bureaucratic barriers whatsoever, so you can submit your first suckless extension in under 20 minutes (includes setup time)