On Fri, 1 Mar 2019, 07:21 Gleb Popov <6year...@gmail.com wrote: > > > On Thu, Feb 28, 2019 at 9:43 PM Ben Cooksley <bcooks...@kde.org> wrote: > >> On Fri, Mar 1, 2019 at 3:13 AM Nate Graham <n...@kde.org> wrote: >> > >> > ---- On Wed, 27 Feb 2019 23:02:03 -0700 Ben Cooksley < >> bcooks...@kde.org> wrote ---- >> > > In terms of server load, it would be nice if the setup of forks was >> > > still something the developer had to initiate rather than being done >> > > automatically for every repository touched by kdesrc-build (I say >> this >> > > mainly as if we had 50 people fork just half of the mainline >> > > repositories we have, that's ~450GB of space used up - a massive >> > > scalability issue) >> > >> > This seems like a challenge that needs to be addressed regardless of >> whether or not kdesrc-build does it automatically, because people creating >> tons and tons of forks is guaranteed to happen anyway if we move to Gitlab. >> It seems non-optimal if having more people able to submit merge requests >> results in the potential to blow up our servers. >> >> We have a little over 1,000 mainline repositories, so in the above >> example we'd be talking about 25,000 forks being created - and i'd be >> expecting quite a bit more than 50 people to use kdesrc-build. To use >> another scenario, if the metric of half the repositories being >> involved (or even a quarter) held true with say 300 users, you're now >> looking at 75,000 - 150,000 forked repositories (and probably around >> 1.4TB - 2.7TB of space used) courtesy of an automated tool. >> >> It would take quite a while for us to reach 150,000 forked >> repositories on Gitlab if humans were to be creating these manually, >> however if an automated tool is going to be creating them as part of >> it's workflow, then we will hit it much more quickly (and is a >> phenomenal waste of resources given virtually all of those forks will >> never be utilised) >> > > I wonder if advanced filesystem features like ZFS deduplication may help > in this situation. >
Deduplication can only do so much. Even if you were to solve the capacity/resource issues, you would hit another issue: Your personal namespace would be flooded with these automatically created forks. This would make your actual personal projects (which are future playground projects in most cases) nearly impossible to find. > >> I certainly do expect a number of forks to be created yes, but i'd >> rather they be useful forks where someone at least intends on working >> on something, rather than ones created automatically by software "just >> in case" someone decides to work on a project. >> >> > >> > Nate >> > >> >> Cheers, >> Ben >> > Regards, Ben >
