On Sat, Mar 5, 2022 at 8:36 AM Ben Cooksley <bcooks...@kde.org> wrote:
> On Fri, Mar 4, 2022 at 12:49 AM Aleix Pol <aleix...@kde.org> wrote: > >> I'd say wireshark is too low level for what the problem is here. We are >> talking about having too many HTTP requests for specific URLs. >> > > Correct, I guess the difference in our approaches comes from a "before > release" to a "monitor after release" angle to things. > I'd like to see increased scrutiny during the development process as well > to make sure that we release code that operates properly from Day 1. > A way to do this could be using commit hooks that do not allow to reach certain services. (which we discussed in private chat). We could also analyse at cmake time the knsrc files we install, but this has a very limited and specific scope. > I can think two main measures: >> - Trigger an alarm (an e-mail notification?) if there's a specific >> UserAgent that has a specific portion of the queries we have in a specific >> day in the services we care about. >> - Offer plots to see how queries by UserAgent evolve over the last couple >> of months (or couple of years). >> > > At the moment our ability to analyse our logs is somewhat limited by our > Privacy Policy - https://kde.org/privacypolicy/ > Currently we don't have any provision for long term storage of > this information even on an aggregated basis - so we would need to update > this first. > Hopefully the NDA should help here and it doesn't seem all that far away. I know Neofytos and Ade have been working on it lately. The second issue there is that we are transitioning users to contact a CDN > based endpoint (which is substantially more scalable). > This does mean we lose visibility on data such as User Agents and the URLs > being impacted though as we only get aggregated data unless we ask for > raw logs - which makes implementing something like what you've described > much harder. > That does seem like a stopper. Still, it seems like it's not that big of a problem when there is a CDN, so we better worry about the other cases. Aleix
