After some discussions with Christian Heimes, it seems this has something to do 
with the latest kernel (4.9), as the kernel's api used in this test changed 
[0][1]. This will need to be addressed in python, but it is weird
that it happens only on specific architectures.

[0] 
https://github.com/smuellerDD/libkcapi/commit/be242c387b7030cbccae2c183107efa86d9a3cd6
[1] http://bugs.python.org/issue29324

Anyway I don't think any more attention is required by infra for this specific 
case.

Regards,

Charalampos Stratakis
Associate Software Engineer
Python Maintenance Team, Red Hat


----- Original Message -----
From: "Kevin Fenzi" <ke...@scrye.com>
To: devel@lists.fedoraproject.org
Sent: Wednesday, January 18, 2017 7:55:38 PM
Subject: Re: Is there something wrong with the Koji builders?

On Wed, 18 Jan 2017 07:22:49 -0500 (EST)
Charalampos Stratakis <cstra...@redhat.com> wrote:

> Python 3 started failing on i686, x86_64 and arm only.
> 
> https://koji.fedoraproject.org/koji/taskinfo?taskID=17313924
> 
> The failures come from the test_socket.py [0], test_aead_aes_gcm [1]
> and more specifically this line [2] with the message: OSError: [Errno
> 22] Invalid argument
> 
> This test is checking Python's interface for the kernel crypto API
> (added in 3.6 [3]).
> 
> [0]
> https://hg.python.org/cpython/file/3.6/Lib/test/test_socket.py#l5473
> [1]
> https://hg.python.org/cpython/file/3.6/Lib/test/test_socket.py#l5472
> [2]
> https://hg.python.org/cpython/file/3.6/Lib/test/test_socket.py#l5497
> [3] http://bugs.python.org/issue27744

Thats nothing to do with the buildsystem. Or at least if it is, it's
not any of the known issues I was working on. 

As far as I know now everything should be back to working/normal. 

There were 3 issues: 

1. The buildvm's were updated and started crashing under load. They
would spew oopses and finally reboot. This would result in builds on
them getting restarted or dying in strange ways. This seems to be
something particular to their hardware/setup that changed in later
4.8.x and early 4.9.x kernels. We are currently running 4.10rc4 on them
and it's been stable (knock on wood). 

2. Sometimes, very rarely dnf would fail downloading packages for the
build root. We have worked around this by passing dnf via koji several
mirrors where it can download from, so if it fails on one it should
fall back to the next and so on. 
https://pagure.io/fedora-infrastructure/issue/5689

3. In some configurations (where there was more than I host to download
from) koji would fail to download the src.rpm correctly and error out. 
We are working around this now by just pointing koji at one place for
these downloads for now until we can get things fixed. 
https://pagure.io/fedora-infrastructure/issue/5694

Sorry for all the instability the last few days. 

If you see any further issues like the above, let me know. 

kevin

_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
_______________________________________________
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Reply via email to