Couple of things: 1. I don't know if the IDE makes use of HTTP ETag and Last-Modified headers and the outbound If-None-Match and If-Modified-Since headers, much less Cache Control headers, but it certainly *should* and would probably be pretty easy to modify if it doesn't. That would trim the HTTP conversation down to 20-40 bytes for 99.999% of requests, where nothing has changed. No idea if the Apache infrastructure distinguishes between a download versus sending a 304 Not Modified response to say "your existing copy is still the latest", but it should. If everyone is constantly downloading the actual file when nothing has changed, that's a bug, and judging from these problems, a serious one.
2. This kind of thing is truly a job for a CDN. In a recent gig, we used CloudFront in front of serving streaming video chopped into thousands of files for HLS streaming. It was shockingly cheap (I remember Akamai's prices way back when). That's how this really ought to be handled. It also seems like something where, the Apache Foundation being a non-profit, a deal might be reached where they "donate" CDN hosting of small things like that, which gets them a tax write-off. Or we could self-fund it - I'd be willing to kick in $100 toward that, which would probably buy a few months of hosting (if the client side actually makes use of HTTP caching headers so we don't blow through what we paid for pointlessly) - if a few other folks were willing to pitch in too. -Tim On Tue, May 14, 2019 at 9:55 AM Bertrand Delacretaz <[email protected]> wrote: > Hi, > > I think something similar happened a few weeks ago already, see here: > > > https://lists.apache.org/thread.html/97f0619de82a630b6ec7d56f1ac5ab34dd85939ae1ff8dcf01434ee9@%3Cusers.infra.apache.org%3E > > NetBeans users going through a company router have been banned from > *.apache.org due to too many requests in a short amount of time to > this URL: > > > http://www.apache.org/dyn/closer.lua?action=download&filename=incubator/netbeans/incubating-netbeans/incubating-11.0/nbms/updates.xml.gz > > It might be worth discussing with infra how to avoid this - my > understanding (to be verified) is that it's the amount of requests to > closer.lua that's problematic, even if the download themselves go to > download mirrors. > > -Bertrand > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > For further information about the NetBeans mailing lists, visit: > https://cwiki.apache.org/confluence/display/NETBEANS/Mailing+lists > > > > -- http://timboudreau.com
