On 1/29/2011 10:00 AM, Mike wrote:
Hello,
My company is small clec / broadband provider serving rural
communities in northern California, and we are the recipient of a
small grant from the state thru our public utilities commission. We
went out to 'middle of nowhere' and deployed adsl2+ in fact (chalk one
up for the good guys!), and now that we're done, our state puc wants
to gather performance data to evaluate the result of our project and
ensure we delivered what we said we were going to. Bigger picture, our
state is actively attempting to map broadband availability and service
levels available and this data will factor into this overall picture,
to be used for future grant/loan programs and other support
mechanisms, so this really is going to touch every provider who serves
end users in the state.
The rub is, that they want to legislate that web based
'speedtest.com' is the ONLY and MOST AUTHORITATIVE metric that trumps
all other considerations and that the provider is %100 at fault and
responsible for making fraudulent claims if speedtest.com doesn't
agree. No discussion is allowed or permitted about sync rates, packet
loss, internet congestion, provider route diversity, end user computer
performance problems, far end congestion issues, far end server issues
or cpu loading, latency/rtt, or the like. They are going to decide
that the quality of any provider service, is solely and exclusively
resting on the numbers returned from 'speedtest.com' alone, period.
All of you in this audience, I think, probably immediately
understand the various problems with such an assertion. Its one of
these situations where - to the uninitiated - it SEEMS LIKE this is
the right way to do this, and it SEEMS LIKE there's some validity to
whats going on - but in practice, we engineering types know it's a far
different animal and should not be used for real live benchmarking of
any kind where there is a demand for statistical validity.
My feeling is that - if there is a need for the state to do
benchmarking, then it outta be using statistically significant
methodologies for same along the same lines as any other benchmark or
test done by other government agencies and national standards bodies
that are reproducible and dependable. The question is, as a hotbutton
issue, how do we go about getting 'the message' across, how do we go
about engineering something that could be considered statistically
relevant, and most importantly, how do we get this to be accepted by
non-technical legislators and regulators?
Mike-
You took the state's money so you are stuck with their dumb rules.
Furthermore the CPUC people aren't stupid. They have highly paid
consultants as well as professors from colleges in California that are
advising them. Unless you have some plan for a very inexpensive
alternative, don't think you are going to make any headway