This comes from the upstream code structure. We've discussed making
custom source tarballs to work around this. The idea is to use the
'repo' tool with a custom manifest.xml to check out just the git repos
that are actually used by the Debian packages. Then use as much of the
upstream build system as possible.