Hello Butch,
Thanks for your insight with this problem. I read through the
information and your blog. I believe we should pursue implementing the
method you mention to try and balance the usage of the users. The
prioritization script is something we want also but at the moment even
http is suffering horribly. We just upgraded the feed to 4Mbps/512Kbps
and it looks almost exactly the same performance wise. Average daily
peak throughput really hasn't gone up much. I was originally told
specifically that we were on a 5:1 oversell, but they just recently
informed us that it is actually a 10:1 oversell 8(. A recent speedtest
run by a user at 8pm showed 220Kbps down and 60Kbps up. At 1am in the
morning (5am EST) when running a test I see the network average around
1200Kbps but it also dips as low as 800Kbps and peaks briefly to
1.9Mbps. On a second test I can only peak the network up to 1100Kbps.
This is very disappointing since this is about the best I can ever
expect to get from this system at this service level, it only goes
downhill or more expensive from here.
We were already looking at upgrading the mikrotik router at the core of
the network. Currently our core unit handles NAT, Web-proxy cache, and
an 8 port serial for dial-up modems. NAT is a mean thing to do but the
satellite provider only allowed us enough IPs for about 20-30% of our
customer base, so we reserve them for businesses. We actually run
Mikrotiks at each base site to provide per user bandwidth limits and for
monitoring.
What I was wondering is with the complexity of NAT and web-proxy would
we be better off running a separate Mikrotik at the core just to handle
the additional bandwidth control we are discussing? I also was thinking
it might be possible that one of the supermicro dual core atom 1U
rackmount servers with hard drives for cache and Mikrotik DOM might be
able to handle everything (except for the modems of course). I haven't
looked into alternatives for modems though, maybe I could just connect
them all through USB if we decide to keep them. I think we are only
using 4 or 5 modems and we are trying to get them moved to 900Mhz.
I believe we would like to hire you to get your assistance to setup the
bandwidth management that we are discussing. I see the prioritization
script is for sale on your store as well. I have done a little bit of
scripting so I was curious what versions of MIkrotik OS does it work on?
- Dan
On 5/21/2010 8:36 PM, Butch Evans wrote:
On Fri, 2010-05-21 at 17:24 -0800, Dan Ferguson wrote:
I haven't checked those out yet, I will have to go take a look. We are
willing to try it, I am sure of that.
I'm sorry, but I missed the first message in this thread. I will try to
address some of your questions/concerns now, though.
We have a site in a very tiny town in McGrath Alaska that we have to run
on Satellite, the amount of bandwidth we receive is always variable.
This will be the most difficult part to deal with. In general, when we
have an unknown amount of bandwidth available to us, the router will not
be able to accurately share that. What we CAN do, though, is ensure
that the "priority" traffic goes first.
The problem with
this is we are unable to effectively shape users to allow all users to
get a slice of the overall feed.
You have very accurately described the problem.
The first couple of customers can reach their
service level and then the rest get stuck in the mud. So a couple will
hit 512Kbps (with one session) and then other may see only 8-12Kbps
(bits).
This is where prioritization comes into play. There are 2 things that
we need to do. First, we can use a queuing algorithm that can help to
"enforce sharing". With a fairly small amount of bandwidth overall, SFQ
may be a good choice. The algorithm that SFQ uses attempts to enforce
sharing by keeping every "flow" equal. In other words, 1 flow gets 100%
of the bandwidth available to the queue, 2 flows would get 50% each.
The problem with this in your situation is that SFQ does not work very
well with an unknown "maximum" availability of bw AND it bases is
algorithm on flows and NOT in IP addresses.
RED queues are an "advancement" from SFQ. Like the SFQ, the RED queue
algorithm will try to enforce sharing, but it manages the flows such
that it tries to keep these flows with an average queue (actual packets
in the queue) equal. In other words, instead of keeping each flow at
the same speed, RED will try to keep the queued packets equal. The
algorithm for RED seems to work better in the situation you have
described when compared to SFQ.
PCQ is probably the best choice for you, but PCQ is an algorithm that
REALLY needs to know the maximum speed available. PCQ is like a "bundle
of FIFO queues". Each of the subqueues (based on IP addresses) is given
it's own FIFO queue and the output side of the queue is managed in such
a way that each subqueue gets the same speed. It is VERY accurate when
the overall bw availability you set is accurate to what is actually
available.
In your situation, I'd probably START with RED and then try PCQ and then
SFQ (if all else failed).
We could lower each users capable service
level but that would only delay the problem from occurring until more
users had peaked the unknown amount of bandwidth available.
With the above tutorial in mind, I will suggest that the other part of
the solution is NOT to limit speeds to the individual user. You are
correct, I think, that it would (at best) just delay the problem. The
other part of the QOS solution that Josh (and others) mentioned is the
scheduling (or prioritization) of traffic. The algorithms above are
just the "means to an end". The actual QOS portion of my script allows
you to easily set priority on specific traffic types. We can enforce
sharing in such a way that short, bursty traffic gets priority over
longer (more resource intensive) downloads. There are a number of ways
to improve how the network responds, but the "problem" you describe is a
VERY hard nut to crack. I really think a good scheduling QOS will help,
though I can't say I've been able to test as well in the specific
circumstance of unknown bandwidth availability.
You can read about the actual methods (well, my thought processes
anyway) on my blog, but I am willing to answer any further questions you
may have either here or in a private exchange. FWIW, there is a link to
my blog at the bottom of every message to this list.
_______________________________________________
Mikrotik mailing list
Mikrotik@mail.butchevans.com
http://www.butchevans.com/mailman/listinfo/mikrotik
Visit http://blog.butchevans.com/ for tutorials related to Mikrotik RouterOS
