Turns out there is a builtin solution that does what I want.
https://wiki.jenkins-ci.org/display/JENKINS/Building+a+matrix+project
And it is mentioned in Jenknis the definitive guide specifically as the
way to go when building jobs running web tests against multiple
browsers/OSs.
Still, one
I've done templated builds using the technique described in this article
and it worked quite well:
http://www.blackpepper.co.uk/generating-new-jenkins-jobs-from-templates-and-parameterised-builds/
For the matrix builds... take a look at the combination filter. This
plug-in might also be useful