I have had this exact same question for my server. It's running on an ADSL line, 443 on the OR port, 80 for the Dir port (iptables redirects to 9001 and 9030) and I notice MUCH more bandwidth usage when I turn on the dir port. Without the dir port, I barely notice tor slowing down my regular usage. With it, I often want to turn off tor to do anything interesting on the net.
I've been using the burst and mac bandwidth settings, but I think prioritizing might be the better way to go. Any good simple references for ip tables? Another thought I had: Does opening up port 80 create problems all by itself? I'm imagining non-tor users simply going to 80 and doing GET / and slurping the results from my tor server, but not really wanting it.