On Wed, 12 Oct 2022 at 07:19, Michal Schorm <msch...@redhat.com> wrote:

> Hello,
> I'm working on a rebase of the MySQL package (community-mysql).
>
> I'm getting:
>   'No space left on device'
> errors very consistently.
> It happens on every single architecture, which is odd as usually such
> problems in KOJI are arch-specific.
>
> It happens very early during testsuite execution.
> Examples here:
>   https://koji.fedoraproject.org/koji/taskinfo?taskID=92917358
>   https://koji.fedoraproject.org/koji/taskinfo?taskID=92919482
>
> I've also tried to re-build the currently released commit, to
> eliminate any changes in behaviour during the package rebase.
> The errors are there too:
>   https://koji.fedoraproject.org/koji/taskinfo?taskID=92945427
>   https://koji.fedoraproject.org/koji/taskinfo?taskID=92945615
>
> Anyone recall any recent changes to KOJI?
>

No there have been no recent changes to the buildsystem. I think the
problem you are running into is the long running one that koji will choose
builders for tasks without knowing what ALL the available resources are on
that box. So if a lot of builds have been going on boxes, you will have
builders with little disk space but koji will still assign tasks to them as
their CPU is empty.


> Would it be possible to allocate more space to the builders ?
>
>
Not without having less builders. Most of the virtual builders split the
CPU/memory/disk space to whatever the base hardware has. That means that
the disk size per system is different because the lower hardware may have
different size. That said the server you ran out of disk space has 100GB of
disk space.

buildvm-s390x-22.s390.fedoraproject.org
/dev/vda2        99G  4.4G   89G   5% /

buildvm-a32-30.iad2.fedoraproject.org
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda5       136G  4.8G  126G   4% /

Other examples to show how disk space can be different
buildvm-a64-30.iad2.fedoraproject.org
# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda3       292G  6.7G  280G   3% /

# df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda2       255G  6.2G  240G   3% /

Builders keep old builds for a certain amount of time and then will get
cleaned up. This means that if a lot of builds are done from various
things, we will run out of disk space on certain builders for a short time.


Michal
>
> --
>
> Michal Schorm
> Software Engineer
> Core Services - Databases Team
> Red Hat
>
> --
> _______________________________________________
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>


-- 
Stephen Smoogen, Red Hat Automotive
Let us be kind to one another, for most of us are fighting a hard battle.
-- Ian MacClaren
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Reply via email to