Hannu Nyman <hannu.ny...@iki.fi> [2023-06-01 19:11:13]: Hi,
> * zabbix x13 (and borgbackup plus other python packages before it) seem to > be the typical last lines before he timeout failures... Random failures or > something due to recent changes ??? IIUC then from the buildbot master perspective it seems like the compilation got stuck somewhere for 1 hour, thats indeed quite strange. I don't remember seeing those errors previously. Anyway, as a quick fix attempt, I've now changed build worker CPU affinity, so each worker uses CPUs from the same NUMA node: NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22 NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23 - cpuset: 0-5 + cpuset: 0,2,4,6,8,10 We've 2 such hosts with 2x Xeon X5680 CPUs, having 4 build workers on each, using 6 CPUs for each build worker. So lets see if it improves the situation, I'll try to look at those build workers more closely now. > * with 23.05 the main problem seems to be "No space left on device"... I've noticed that as well, probably 23.05 is more disk space greedy and current 60GB allocation is not enough, so going to increase the disk size to fix that. I've just added 2x8 temporary build workers to cleanup the build queue and refresh the packages. Cheers, Petr _______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
