Well, I've got it. It turns out it's kind of easy, although not as pretty as it could be.
Basically, you use relayd. The one caveat is that this means that from the OpenBSD box, you need to be able to talk to the remote, private IPs without binding to a particular address. In relayd.conf, you enable relays on a port-by-port basis, so you can't allow blanket access.
