On Thu, Sep 23, 2021 at 11:52 AM Ben Cooksley <bcooks...@kde.org> wrote: > > Hi all, > > It has recently come to our attention that the number of queries being > handled for the endpoint https://autoconfig.kde.org/ocs/providers.xml on a > day to day basis has gotten to the point where it is causing issues with > server responsiveness to other traffic. This is perhaps best summarised by > the following: > > root@nicoda /var/log/apache2 # ls -lah ... > -rw-r----- 1 root adm 458M Sep 23 06:25 autoconfig.kde.org.log.1 > -rw-r----- 1 root adm 381M Sep 23 06:25 networkcheck.kde.org.log.1 > -rw-r----- 1 root adm 143M Sep 23 06:25 www.kde.org.log.1 > > root@nicoda /var/log/apache2 # cat autoconfig.kde.org.log.1 | wc -l > 4,222,343 > > Based on those numbers we're looking at 48-49 requests per second (on average > - peaks are much higher by many magnitudes), which seems extremely excessive > given that this file is only supposed to be retrieved by KDE software when > GHNS functionality is triggered. That is supported by the substantial size > difference it has over networkcheck.kde.org - which is used by plasma-nm and > NetworkManager (on Neon) to check for whether they have a working internet > connection - which i'd expect to be the site receiving the most traffic. > > As such, I therefore suspect we have bug(s) in software that makes use of > GHNS functionality. > > It would therefore be appreciated if we could please review the software in > question to determine whether it is operating correctly. Given that it > usually runs in the background on user systems, i'd especially appreciate it > if a detailed review could be conducted on Discover and other software that > conducts package management operations or assists in managing updates. > > Unfortunately all these applications submit a fairly useless user agent > (Mozilla/5.0) so it is impossible for Sysadmin to ascertain any further > information. If we could get information on the software that is originating > the request added to the user agent to assist in investigating these issues > in the future that would be extremely helpful. > > Thanks, > Ben
That's correct. Discover fetches them at startup. It's necessary to be able to check if there are updates on KNS-provided resources. Incidentally, I was looking into this yesterday incidentally. We could see if caching is broken somehow. A request will still be needed though to check if the cache is out of date. Or we can prefer the cached version of the file for a few days and assume they don't change that much. I've seen some of them point at "https://download.kde.org/ocs/providers.xml";, I don't know if this is any better. Aleix
