While testing pfstat, I noticed that FreeBSD's port of gd 2.0.33 [1] has a WITHOUT_X11 "flavor". It does pull in freetype2, but that doesn't depend on X11 there.
No, I don't have a patch yet ;) Daniel [1] http://www.freebsd.org/cgi/cvsweb.cgi/ports/graphics/gd/Makefile?rev=1.78