Hi,
On Fri, 13 Apr 2012, Peter Pöml wrote:
Am 03.04.2012 um 18:17 schrieb Roberto Galoppini:
We at SourceForge have worked the last ten days to line-up dedicated
infrastructure (including CDN services) to support the upcoming AOO
download serving test.
I can hardly believe reading this!
Me too. What an ignorance of proven and waiting mechanisms.
What's going on? We have an existing (and well working) mirror network,
that handles any required load just fine. It's proven and time-tested.
It has survived all releases with ease. By all calculation, and by
practical experience, the combined upload capacity of the mirrors is
sufficient to satisfy the peak download demand as well as the sustained
demand. By the way, the "peak download demand" doesn't really differ a
lot from the day-to-day download demand, contrary to public belief. The
mirrors are numerous and spread around the world, and the chance of a
client being sent to a close and fast mirror is good - better than with
a handful of mirrors as is the case with the Sourceforge mirror network.
Sourceforge specializes in something different - providing a myriad of
small files by a set of specialized mirrors. "Normal", plain simple
mirrors can't take part in this network as far as I can tell.
Yes. I had tried to help with ftp5.gwdg.de - impossible "unconditionally".
Even though the network was considerably extended a few years ago, from
10 (under 10?) to >20 mirrors, this is still a small number of mirrors.
(Even though these are power-mirrors, but those are part of our existing
mirror network just as well.)
With our mirror network, mirrors can mirror partial content, so they can
provide what's important in their region, like certain language packs
only. This greatly increases the likelyhood of finding mirrors in remote
areas, that don't have hundreds of gigabytes to spare. It's also
unnecessary that mirrors carry old releases that are infrequently
downloaded. Mirrors can run whatever HTTP software they prefer, not only
Apache httpd, or even FTP servers. Mirrors can decide to offer mirroring
only in their network/autonomous system/country to limit the share of
requests they get, and from where they get it. Many mirrors don't have
good international connectivity, but can be used well with us
nevertheless. We provide cryptohashes, Metalinks, even P2P links, all
fully automatically. That's very important for these unusually large
files. Downloading without error correction is not fun. We select
mirrors by GeoIP, but also by geographical distance as well as network
topology, whatever gives a close match, and we already support IPv6.
It has taken some years to build all this, and a lot of the features
were triggered directly by the work on the OpenOffice.org redirector.
Built for OpenOffice.org
The software is the one kind of work that went into it, finding and
collecting mirrors the other thing, building trust and lasting
relationship. A mirror network isn't built overnight.
I think there is a danger that the Apache mirror network is equated with
the OOo mirror network. This is a mistake in my view. The large files
that we have are a totally different challenge. It's a huge difference
to download 6MB tarballs and 200MB files, both from the users
perspective ("why does my file not work, that I waited so long for!?")
and from the mirrors perspective ("what are these 200 connections from
Chinese IPs on my mirror server!?"). It is important to be able to give
mirrors different weight, because they differ vastly in their
capabilities, which can range from 4GBit bandwidth down brittle to
50Mbit somewhere else. Even inside an "Internet country" like Germany
you'll have differences of 100 MBit to multiple Gbit, and you want to
utilize the bandwidth well. We have this working well!
I can confirm this, I have watched the growing "intelligence" of
MirrorBrain from the beginning.
OpenOffice.org used a software called "Bouncer" before switching to
MirrorBrain, which was one of the simpler solutions. I think everybody
(who has been in the project a few years) will agree that we don't want
to go back.
Surely. The OpenOffice step from bouncer to mirrorbrain was all over
agreed a performance and quality step.
BTW, dear Apache people, I am the one that helped StarOffice Hamburg to
publish their first opensource release - maintainer of ftp.gwdg.de since
20 years.
So I see that Sourceforge wants to beef up their network by renting a
Content Delivery Network (CDN). Is that needed? yes, because they don't
have enough bandwidth in mirrors. Is that a good idea? I don't think so,
but I'm biased, because 1) I don't like advertisements and 2) I'm
strongly rooted in the mirror community with both legs.
Didn't mirrorbrain lately help Novell to save a lot of money they
regularly had spended to Akamai before? I guess it was this way.
In the mirror community, there is a kind of self esteem among the more
ambitious mirror admins: they believe that stepping in of commercial
CDNs is not needed to handle even peak download demand of the most
popular Open Source software. And they work hard for it.
Yes, we do. All mirror admins love to see their lines full. That is the
temporary excitement we are struggling for. Mirrorbrain can give us this
picture at the spot moments without frustrating any single user.
Together, we have proven that the help of commercial CDNs is *not*
needed, both with OpenOffice.org and with OpenSUSE.org. Mirrors have
served > 20 GByte per second together. The bandwidth is there! (In the
past, Akamai was used during release peaks with OpenSUSE.org, so I have
been there, and also got interesting insight and numbers there.)
I tried the currently configured download from
http://www.openoffice.org/download today (from a real crappy end user
box ;). It was slow and didn't start downloading immediately, but showed
a page full of advertisement that didn't have any relation to
OpenOffice.org, wanted to open a popup (MS IE said that and blocked it)
Hey, Peter, you and MS IE - what's going on? Are you letting others to
drive you crazy?
and when the download started, it came from the Swiss mirror, but I'm in
Germany! What's that? Thrown 3 years back in time? Sub-optimal. (I can
guess who pays for the CDN that is rented to help out: advertising.)
Do you really want to ditch what we have built? Ditching the system that
improved downloading OpenOffice.org in the farthest corners of the
world? Exchanging it against a handful of Sourceforge mirrors, and 250
Apache mirrors, many of which lack the capability? Some are big, but
many will be far from having the bandwidth to deliver large files.
Something that Apache's mirror system also can't do is sending me to my
local mirror (my very ISP in my city runs a mirror, and my home IP is in
their netblock). Apache mirror system sends me to *any* mirror in my
country, while our current solution recognizes the network topology and
lets me download from the local mirror. Especially with large files,
that's very nice both for the ISP and for me as user. Sourceforge can
theoretically do this (because they use a part of MirrorBrain for that
purpose!) but don't have enough mirrors to play this out. This is not
only useful with single ISPs, if they have a mirror; it's also useful
with autonomous systems (AS) of networks that share a backbone, like
most German universities in AS680 here in Germany.
The german university network (DFN-Verein, some members already are
"producing" 10 gbit) was the base infrastucture for the openoffice
spreading (and staroffice before, and is now already with libreoffice
too).
Please don't neglect this chance for the Apache Foundation. It clearly is
offered (and - regarding ftp.gwdg.de and many more - since the beginning
of Apache practized).
So we will have a *technically inferiour* solution in the future? That's
not the Apache way, is it?
I have been told more than once, on this list, that "it will be the
Apache mirror system and nothing else". I didn't understand the reasons
(except for policy, no special treatment for individual projects), but
it won't work that way IMO.
Now it seems to me that the Apache mirror system seeked the help of
Sourceforge.net. If that means that some doubts crept up, then I share
those doubts. But I don't see Sourceforge.net as the solution either, as
explained above. They have their merits, and I like their dedication and
the specialized system they've built (with features that I'm envious
of!), but I think our existing solution is better suited. And not only
that, IMO it is a very important prerequisite of being successful. No
well-working downloads, no luck with distributing FOSS that consists of
large files.
Dear Apache Foundation, please listen to Peter's words and use his work.
It will be a win for you - incredible that you did not realize that
already by yourself. You are a "community product", and so you should help
to show that "the community" is autonomous.
Viele Gruesse
Eberhard Moenkeberg (emoe...@gwdg.de, e...@kki.org)
--
Eberhard Moenkeberg
Arbeitsgruppe IT-Infrastruktur
E-Mail: emoe...@gwdg.de Tel.: +49 (0)551 201-1551
-------------------------------------------------------------------------
Gesellschaft fuer wissenschaftliche Datenverarbeitung mbH Goettingen (GWDG)
Am Fassberg 11, 37077 Goettingen
URL: http://www.gwdg.de E-Mail: g...@gwdg.de
Tel.: +49 (0)551 201-1510 Fax: +49 (0)551 201-2150
Geschaeftsfuehrer: Prof. Dr. Ramin Yahyapour
Aufsichtsratsvorsitzender: Prof. Dr. Christian Griesinger
Sitz der Gesellschaft: Goettingen
Registergericht: Goettingen Handelsregister-Nr. B 598
-------------------------------------------------------------------------
