OK, so having researched this a little more, it seems that what I really
want is to put the reply-to option on either the firewall rule or the
NAT rule (haven't quite figured out which one of those it goes on.) as
per from the manual page of pf.conf(5):
/reply-to/
The/reply-to/ option is similar to/route-to/, but routes packets that
pass in the opposite direction (replies) to the specified inter-
face. Opposite direction is only defined in the context of a state
entry, and/reply-to/ is useful only in rules that create state. It
can be used on systems with multiple external connections to route
all outgoing packets of a connection through the interface the
incoming connection arrived through (symmetric routing enforce-
ment).
You can do route-to by using the "gateway" dropdown, but a "reply-to"
dropdown seems to be missing. This would seem to be an option that needs
to be added to the web UI if I want this to work, unless I'm missing
something.