[us...@bb.net] And yet even more anecdotes...

2018-08-15 Thread Neil Gilmore

Hi everyone,

Mostly, our system has been running well. This summer, we did have an 
intern who we gave the job of both updating our masters and making our 
UI changes work on the latest version. So we're currently running 1.2.0 
for our masters. We're further behind on the workers, because updating 
those is painful for us.


We did run into an odd problem over the weekend. As you might remember, 
we divide our builders essentially two ways. The smaller set produces 
our installers. The larger set uses those installers to run tests. Our 
installer builders use locks to make sure that only one build is running 
at a time. We can't use the usual mechanism because we also have one 
builder for monitoring that needs to run concurrently with the 'active' 
build.


We changed from using one branch to another. This was done after the old 
branch's builds started. Because we construct builder names from the 
branch names, this can mean that we create new builders, or resurrect 
older builders (which is what happened in this case). Naturally, we no 
longer see the old  builders by default.


Every one of those builders got stuck acquiring locks. We do see this 
problem from time to time, but it's usually a single host's builds.


Cancelling the build won't solve the problem. The next build will also 
get stuck. Restarting the worker doesn't help, nor does a 
reconfiguration. Restarting the master will solve the problem, but 
that's pretty drastic, and we tend to lose a lot of work when we do that.


If this happens to you, and you need to fix it, here's what we do. We 
use the manhole in twistd. And we do these:


foo = master.botmaster.namedServices['builder>'].building[0].locks[0][0]

foo.release(foo.owners[0][0], foo.owners[0][1])

The index of 'building' might not always be 0, but almost always is.

I remember some discussion of whether newer versions of twistd still had 
the manhole. Because we need to do this regularly (though thankfully not 
frequently), I was dreading updating twistd. But it looks like it survives.


Thanks for your time reading this!

Neil Gilmore
grammatech.com
___
users mailing list
users@buildbot.net
https://lists.buildbot.net/mailman/listinfo/users

[us...@bb.net] warningPattern: scope of the search

2018-08-15 Thread Martin Liška
Hello.

I've got a build step where I do a grepping:

```
find gcc/testsuite -name "*.log" | xargs cat | grep -v "Fortran runtime error:" 
| grep -v "fortran/simplify.c:" | grep -B10 -A10 "runtime error:" || true
 in dir /home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir 
(timeout 1200 secs)
 watching logfiles {}
 argv: b'find gcc/testsuite -name "*.log" | xargs cat | grep -v "Fortran 
runtime error:" | grep -v "fortran/simplify.c:" | grep -B10 -A10 "runtime 
error:" || true'
 environment:
  DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
  HOME=/home/marxin
  LC_CTYPE=en_US.UTF-8
  LOGNAME=marxin
  PATH=/usr/bin:/bin
  PWD=/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir
  SHELL=/bin/sh
  SHLVL=1
  USER=marxin
  XDG_RUNTIME_DIR=/run/user/1000
  XDG_SESSION_ID=3
  _=/usr/bin/buildbot-worker
 using PTY: False
PASS: gcc.dg/strlenopt-53.c scan-tree-dump-times gimple "strlen" 0
PASS: gcc.dg/strlenopt-53.c scan-tree-dump-times ccp1 
"call_in_true_branch_not_eliminated" 0
Executing on host: 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/xgcc 
-B/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/ 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/testsuite/gcc.dg/strlenopt-54.c
-fno-diagnostics-show-caret -fno-diagnostics-show-line-numbers 
-fdiagnostics-color=never   -O2 -Wall -fdump-tree-optimized -S -o 
strlenopt-54.s(timeout = 300)
spawn -ignore SIGHUP 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/xgcc 
-B/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/ 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/testsuite/gcc.dg/strlenopt-54.c
 -fno-diagnostics-show-caret -fno-diagnostics-show-line-numbers 
-fdiagnostics-color=never -O2 -Wall -fdump-tree-optimized -S -o strlenopt-54.s
PASS: gcc.dg/strlenopt-54.c (test for excess errors)
PASS: gcc.dg/strlenopt-54.c scan-tree-dump-times optimized "strlen" 0
PASS: gcc.dg/strlenopt-54.c scan-tree-dump-times optimized 
"call_in_true_branch_not_eliminated" 0
Executing on host: 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/xgcc 
-B/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/ 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/testsuite/gcc.dg/strlenopt-55.c
-fno-diagnostics-show-caret -fno-diagnostics-show-line-numbers 
-fdiagnostics-color=never   -O1 -Wall -fdump-tree-gimple -fdump-tree-optimized 
-S -o strlenopt-55.s(timeout = 300)
spawn -ignore SIGHUP 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/xgcc 
-B/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/ 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/testsuite/gcc.dg/strlenopt-55.c
 -fno-diagnostics-show-caret -fno-diagnostics-show-line-numbers 
-fdiagnostics-color=never -O1 -Wall -fdump-tree-gimple -fdump-tree-optimized -S 
-o strlenopt-55.s
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/testsuite/gcc.dg/strlenopt-55.c:127:3:
 warning: unsigned conversion from 'int' to 'short unsigned int' changes value 
from '65536' to '0' [-Woverflow]
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/poly-int.h:1095:5:
 runtime error: signed integer overflow: 9223372036854775805 * 8 cannot be 
represented in type 'long int'
PASS: gcc.dg/strlenopt-55.c  (test for warnings, line 127)
FAIL: gcc.dg/strlenopt-55.c (test for excess errors)
Excess errors:
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/build/gcc/poly-int.h:1095:5:
 runtime error: signed integer overflow: 9223372036854775805 * 8 cannot be 
represented in type 'long int'
PASS: gcc.dg/strlenopt-55.c scan-tree-dump-times gimple "strlen" 0
PASS: gcc.dg/strlenopt-55.c scan-tree-dump-times gimple "memcmp" 0
PASS: gcc.dg/strlenopt-55.c scan-tree-dump-times gimple "strcmp" 0
PASS: gcc.dg/strlenopt-55.c scan-tree-dump-times optimized 
"call_in_true_branch_not_eliminated" 0
Executing on host: 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/xgcc 
-B/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/ 
IBM10478573.c-fno-diagnostics-show-caret -fno-diagnostics-show-line-numbers 
-fdiagnostics-color=never-lm  -o IBM10478573.exe(timeout = 300)
spawn -ignore SIGHUP 
/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/xgcc 
-B/home/marxin/BIG/buildbot/buildworker/marxinbox-gcc-ubsan/objdir/gcc/ 
IBM10478573.c -fno-diagnostics-show-caret -fno-diagnostics-show-line-numbers 
-fdiagnostics-color=never -lm -o IBM10478573.exe
Setting LD_LIBRARY_PATH to