Re: Go patch committed: New lock/note implementation
Rainer Orth writes: > FAIL: go.go-torture/execute/go-1.go execution, -O0 There should be more information in gcc/testsuite/go/go.log. > * All 64-bit libgo tests fail on the same target: > > FAIL: asn1 > /vol/gcc/src/hg/trunk/local/libgo/testsuite/gotest[422]: gotest-timeout: > cannot create [No such file or directory] > checkId: 65 should be 1 > checkId: 66 should be 1 > > I've not yet investigated what's going on here. I don't know what is going on either. gotest-timeout is just a file created by the gotest script: (sleep `expr $timeout + 10` echo > gotest-timeout echo "timed out in gotest" 1>&2 kill -9 $pid) & The error above implies that gotest is running in a directory which was removed while the script was running. I don't really see how libgo could cause that to happen. Could something else have been going on? At this point I have no idea what is going wrong. Ian
Re: Go patch committed: New lock/note implementation
Ian Lance Taylor writes: >> ... and also Solaris 8 and 9 bootstrap which lack sem_timedwait: >> >> /vol/gcc/src/hg/trunk/local/libgo/runtime/thread-sema.c: In function >> 'runtime_semasleep': >> /vol/gcc/src/hg/trunk/local/libgo/runtime/thread-sema.c:42:7: error: >> implicit declaration of function 'sem_timedwait' >> [-Werror=implicit-function-declaration] > > This one was somewhat trickier, but I think this patch will do the job. > This uses pthread_cond_timedwait instead of sem_timedwait (I hope that > Solaris 8 and 9 have pthread_cond_timedwait). Bootstrapped and ran Go They do, as does IRIX. > testsuite on x86_64-unknown-linux-gnu. Committed to mainline. Together with the sigignore -> sig_ignore change, I'm back to bootstrap land again,thanks. Unfortunately, I'm now (as of r181837) seeing considerable testsuite regressions on go and libgo test results: * many 64-bit go execution failures on i386-pc-solaris2.1[01]: Running target unix/-m64 FAIL: go.go-torture/execute/go-1.go execution, -O0 FAIL: go.go-torture/execute/go-1.go execution, -O1 FAIL: go.go-torture/execute/go-1.go execution, -O2 FAIL: go.go-torture/execute/go-1.go execution, -O2 -fomit-frame-pointer -finline-functions FAIL: go.go-torture/execute/go-1.go execution, -O2 -fomit-frame-pointer -finline-functions -funroll-loops FAIL: go.go-torture/execute/go-1.go execution, -O2 -fbounds-check FAIL: go.go-torture/execute/go-1.go execution, -O3 -g FAIL: go.go-torture/execute/go-1.go execution, -Os FAIL: go.go-torture/execute/go-2.go execution, -O0 FAIL: go.go-torture/execute/go-2.go execution, -O1 FAIL: go.go-torture/execute/go-2.go execution, -O2 FAIL: go.go-torture/execute/go-2.go execution, -O2 -fomit-frame-pointer -finline-functions FAIL: go.go-torture/execute/go-2.go execution, -O2 -fomit-frame-pointer -finline-functions -funroll-loops FAIL: go.go-torture/execute/go-2.go execution, -O2 -fbounds-check FAIL: go.go-torture/execute/go-2.go execution, -O3 -g FAIL: go.go-torture/execute/go-2.go execution, -Os FAIL: go.go-torture/execute/go-3.go execution, -O0 FAIL: go.go-torture/execute/go-3.go execution, -O1 FAIL: go.go-torture/execute/go-3.go execution, -O2 FAIL: go.go-torture/execute/go-3.go execution, -O2 -fomit-frame-pointer -finline-functions FAIL: go.go-torture/execute/go-3.go execution, -O2 -fomit-frame-pointer -finline-functions -funroll-loops FAIL: go.go-torture/execute/go-3.go execution, -O2 -fbounds-check FAIL: go.go-torture/execute/go-3.go execution, -O3 -g FAIL: go.go-torture/execute/go-3.go execution, -Os FAIL: go.go-torture/execute/select-1.go execution, -O0 FAIL: go.go-torture/execute/select-1.go execution, -O1 FAIL: go.go-torture/execute/select-1.go execution, -O2 FAIL: go.go-torture/execute/select-1.go execution, -O2 -fomit-frame-pointer -finline-functions FAIL: go.go-torture/execute/select-1.go execution, -O2 -fomit-frame-pointer -finline-functions -funroll-loops FAIL: go.go-torture/execute/select-1.go execution, -O2 -fbounds-check FAIL: go.go-torture/execute/select-1.go execution, -O3 -g FAIL: go.go-torture/execute/select-1.go execution, -Os FAIL: go.test/test/235.go execution, -O2 -g FAIL: go.test/test/bigalg.go execution, -O2 -g FAIL: go.test/test/chan/doubleselect.go execution, -O2 -g FAIL: go.test/test/chan/fifo.go execution, -O2 -g FAIL: go.test/test/chan/goroutines.go execution, -O2 -g FAIL: go.test/test/chan/nonblock.go execution, -O2 -g FAIL: go.test/test/chan/powser1.go execution, -O2 -g FAIL: go.test/test/chan/powser2.go execution, -O2 -g FAIL: go.test/test/chan/select2.go execution, -O2 -g FAIL: go.test/test/chan/select3.go execution, -O2 -g FAIL: go.test/test/chan/select6.go execution, -O2 -g FAIL: go.test/test/chan/sieve1.go execution, -O2 -g FAIL: go.test/test/chan/sieve2.go execution, -O2 -g FAIL: go.test/test/closure.go execution, -O2 -g FAIL: go.test/test/escape.go execution, -O2 -g FAIL: go.test/test/fixedbugs/bug067.go execution, -O2 -g FAIL: go.test/test/fixedbugs/bug130.go execution, -O2 -g FAIL: go.test/test/fixedbugs/bug147.go execution, -O2 -g FAIL: go.test/test/fixedbugs/bug243.go execution, -O2 -g FAIL: go.test/test/func5.go execution, -O2 -g FAIL: go.test/test/goprint.go execution, -O2 -g FAIL: go.test/test/ken/chan.go execution, -O2 -g FAIL: go.test/test/ken/chan1.go execution, -O2 -g FAIL: go.test/test/ken/cplx5.go execution, -O2 -g FAIL: go.test/test/mallocfin.go execution, -O2 -g FAIL: go.test/test/nil.go execution, -O2 -g FAIL: go.test/test/range.go execution, -O2 -g FAIL: go.test/test/stack.go execution, -O2 -g * All 64-bit libgo tests fail on the same target: FAIL: asn1 /vol/gcc/src/hg/trunk/local/libgo/testsuite/gotest[422]: gotest-timeout: cannot create [No such file or directory] checkId: 65 should be 1 checkId: 66 should be 1 I've not yet investigated what's going on here. Rainer -- --
Re: Go patch committed: New lock/note implementation
Rainer Orth writes:
> Rainer Orth writes:
>
>> This broke bootstrap on Linux/x86_64 (CentOS 5.5), which lacks
>> O_CLOEXEC.
>
> ... and also Solaris 8 and 9 bootstrap which lack sem_timedwait:
>
> /vol/gcc/src/hg/trunk/local/libgo/runtime/thread-sema.c: In function
> 'runtime_semasleep':
> /vol/gcc/src/hg/trunk/local/libgo/runtime/thread-sema.c:42:7: error: implicit
> declaration of function 'sem_timedwait'
> [-Werror=implicit-function-declaration]
This one was somewhat trickier, but I think this patch will do the job.
This uses pthread_cond_timedwait instead of sem_timedwait (I hope that
Solaris 8 and 9 have pthread_cond_timedwait). Bootstrapped and ran Go
testsuite on x86_64-unknown-linux-gnu. Committed to mainline.
Ian
diff -r cf5fa6d457da libgo/configure.ac
--- a/libgo/configure.ac Tue Nov 29 11:38:05 2011 -0800
+++ b/libgo/configure.ac Tue Nov 29 13:54:16 2011 -0800
@@ -456,6 +456,14 @@
AM_CONDITIONAL(HAVE_STRERROR_R, test "$ac_cv_func_strerror_r" = yes)
AM_CONDITIONAL(HAVE_WAIT4, test "$ac_cv_func_wait4" = yes)
+CFLAGS_hold="$CFLAGS"
+CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
+LIBS_hold="$LIBS"
+LIBS="$LIBS $PTHREAD_LIBS"
+AC_CHECK_FUNCS(sem_timedwait)
+CFLAGS="$CFLAGS_hold"
+LIBS="$LIBS_hold"
+
AC_CACHE_CHECK([for __sync_bool_compare_and_swap_4],
[libgo_cv_func___sync_bool_compare_and_swap_4],
[AC_LINK_IFELSE([
diff -r cf5fa6d457da libgo/runtime/lock_sema.c
--- a/libgo/runtime/lock_sema.c Tue Nov 29 11:38:05 2011 -0800
+++ b/libgo/runtime/lock_sema.c Tue Nov 29 13:54:16 2011 -0800
@@ -32,9 +32,11 @@
void
runtime_lock(Lock *l)
{
+ M *m;
uintptr v;
uint32 i, spin;
+ m = runtime_m();
if(m->locks++ < 0)
runtime_throw("runtime_lock: lock count");
@@ -91,7 +93,7 @@
uintptr v;
M *mp;
- if(--m->locks < 0)
+ if(--runtime_m()->locks < 0)
runtime_throw("runtime_unlock: lock count");
for(;;) {
@@ -144,6 +146,9 @@
void
runtime_notesleep(Note *n)
{
+ M *m;
+
+ m = runtime_m();
if(m->waitsema == 0)
m->waitsema = runtime_semacreate();
if(!runtime_casp(&n->waitm, nil, m)) { // must be LOCKED (got wakeup)
@@ -158,6 +163,7 @@
void
runtime_notetsleep(Note *n, int64 ns)
{
+ M *m;
M *mp;
int64 deadline, now;
@@ -166,6 +172,7 @@
return;
}
+ m = runtime_m();
if(m->waitsema == 0)
m->waitsema = runtime_semacreate();
diff -r cf5fa6d457da libgo/runtime/thread-sema.c
--- a/libgo/runtime/thread-sema.c Tue Nov 29 11:38:05 2011 -0800
+++ b/libgo/runtime/thread-sema.c Tue Nov 29 13:54:16 2011 -0800
@@ -2,6 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
+#include "config.h"
#include "runtime.h"
#include
@@ -9,19 +10,43 @@
#include
#include
+/* If we don't have sem_timedwait, use pthread_cond_timedwait instead.
+ We don't always use condition variables because on some systems
+ pthread_mutex_lock and pthread_mutex_unlock must be called by the
+ same thread. That is never true of semaphores. */
+
+struct go_sem
+{
+ sem_t sem;
+
+#ifndef HAVE_SEM_TIMEDWAIT
+ int timedwait;
+ pthread_mutex_t mutex;
+ pthread_cond_t cond;
+#endif
+};
+
/* Create a semaphore. */
uintptr
runtime_semacreate(void)
{
- sem_t *p;
+ struct go_sem *p;
/* Call malloc rather than runtime_malloc. This will allocate space
on the C heap. We can't call runtime_malloc here because it
could cause a deadlock. */
- p = malloc (sizeof (sem_t));
- if (sem_init (p, 0, 0) != 0)
+ p = malloc (sizeof (struct go_sem));
+ if (sem_init (&p->sem, 0, 0) != 0)
runtime_throw ("sem_init");
+
+#ifndef HAVE_SEM_TIMEDWAIT
+ if (pthread_mutex_init (&p->mutex, NULL) != 0)
+runtime_throw ("pthread_mutex_init");
+ if (pthread_cond_init (&p->cond, NULL) != 0)
+runtime_throw ("pthread_cond_init");
+#endif
+
return (uintptr) p;
}
@@ -30,26 +55,56 @@
int32
runtime_semasleep (int64 ns)
{
+ M *m;
+ struct go_sem *sem;
int r;
+ m = runtime_m ();
+ sem = (struct go_sem *) m->waitsema;
if (ns >= 0)
{
+ int64 abs;
struct timespec ts;
+ int err;
- ns += runtime_nanotime ();
- ts.tv_sec = ns / 10LL;
- ts.tv_nsec = ns % 10LL;
- r = sem_timedwait ((sem_t *) m->waitsema, &ts);
+ abs = ns + runtime_nanotime ();
+ ts.tv_sec = abs / 10LL;
+ ts.tv_nsec = abs % 10LL;
+
+ err = 0;
+
+#ifdef HAVE_SEM_TIMEDWAIT
+ r = sem_timedwait (&sem->sem, &ts);
if (r != 0)
+ err = errno;
+#else
+ if (pthread_mutex_lock (&sem->mutex) != 0)
+ runtime_throw ("pthread_mutex_lock");
+
+ while ((r = sem_trywait (&sem->sem)) != 0)
{
- if (errno == ETIMEDOUT || errno == EINTR)
+ r = pthread_cond_timedwait (&sem->cond, &sem->mutex, &ts);
+ if (r != 0)
+ {
+ err = r;
+ break;
+ }
+ }
+
+ if (pthread_mutex_unlock (&sem->mutex) != 0)
+ runtime_throw ("pthread_mutex_unlock");
+#endif
+
+ if (err != 0)
+ {
+ if (err == ETIMEDOUT || err == EAGAIN || err == E
Re: Go patch committed: New lock/note implementation
Rainer Orth writes:
> Ian Lance Taylor writes:
>
>> This patch updates the implementations of locks and notes used in libgo
>> to use the current version from the master Go library. This now uses
>> futexes when running on GNU/Linux, while still using semaphores on other
>> systems. This implementation should be faster, and does not require
>> explicit initialization. Bootstrapped and ran Go testsuite on
>> x86_64-unknown-linux-gnu. I tested both the futex and the semaphore
>> versions. Committed to mainline.
>
>> +static int32
>> +getproccount(void)
>> +{
>> +int32 fd, rd, cnt, cpustrlen;
>> +const byte *cpustr, *pos;
>> +byte *bufpos;
>> +byte buf[256];
>> +
>> +fd = open("/proc/stat", O_RDONLY|O_CLOEXEC, 0);
>
> This broke bootstrap on Linux/x86_64 (CentOS 5.5), which lacks
> O_CLOEXEC.
Thanks for the report. Fixed in the obvious way as follows. The use of
O_CLOEXEC provides extra safety but is not essential, since the program
is single-threaded when the file is used. Bootstrapped on
x86_64-unknown-linux-gnu. Committed to mainline.
Ian
diff -r 295f86b18961 libgo/runtime/thread-linux.c
--- a/libgo/runtime/thread-linux.c Tue Nov 29 11:02:01 2011 -0800
+++ b/libgo/runtime/thread-linux.c Tue Nov 29 11:24:45 2011 -0800
@@ -62,6 +62,10 @@
*(int32*)0x1006 = 0x1006;
}
+#ifndef O_CLOEXEC
+#define O_CLOEXEC 0
+#endif
+
static int32
getproccount(void)
{
Re: Go patch committed: New lock/note implementation
Rainer Orth writes: > This broke bootstrap on Linux/x86_64 (CentOS 5.5), which lacks > O_CLOEXEC. ... and also Solaris 8 and 9 bootstrap which lack sem_timedwait: /vol/gcc/src/hg/trunk/local/libgo/runtime/thread-sema.c: In function 'runtime_semasleep': /vol/gcc/src/hg/trunk/local/libgo/runtime/thread-sema.c:42:7: error: implicit declaration of function 'sem_timedwait' [-Werror=implicit-function-declaration] Rainer -- - Rainer Orth, Center for Biotechnology, Bielefeld University
Re: Go patch committed: New lock/note implementation
Ian Lance Taylor writes:
> This patch updates the implementations of locks and notes used in libgo
> to use the current version from the master Go library. This now uses
> futexes when running on GNU/Linux, while still using semaphores on other
> systems. This implementation should be faster, and does not require
> explicit initialization. Bootstrapped and ran Go testsuite on
> x86_64-unknown-linux-gnu. I tested both the futex and the semaphore
> versions. Committed to mainline.
> +static int32
> +getproccount(void)
> +{
> + int32 fd, rd, cnt, cpustrlen;
> + const byte *cpustr, *pos;
> + byte *bufpos;
> + byte buf[256];
> +
> + fd = open("/proc/stat", O_RDONLY|O_CLOEXEC, 0);
This broke bootstrap on Linux/x86_64 (CentOS 5.5), which lacks
O_CLOEXEC.
Rainer
--
-
Rainer Orth, Center for Biotechnology, Bielefeld University
Go patch committed: New lock/note implementation
This patch updates the implementations of locks and notes used in libgo
to use the current version from the master Go library. This now uses
futexes when running on GNU/Linux, while still using semaphores on other
systems. This implementation should be faster, and does not require
explicit initialization. Bootstrapped and ran Go testsuite on
x86_64-unknown-linux-gnu. I tested both the futex and the semaphore
versions. Committed to mainline.
Ian
diff -r 8b1402d2f0d3 libgo/Makefile.am
--- a/libgo/Makefile.am Tue Nov 22 12:16:53 2011 -0800
+++ b/libgo/Makefile.am Tue Nov 22 12:18:39 2011 -0800
@@ -394,6 +394,12 @@
rtems_task_variable_add_file =
endif
+if LIBGO_IS_LINUX
+runtime_lock_files = runtime/lock_futex.c runtime/thread-linux.c
+else
+runtime_lock_files = runtime/lock_sema.c runtime/thread-sema.c
+endif
+
runtime_files = \
runtime/go-append.c \
runtime/go-assert.c \
@@ -432,7 +438,6 @@
runtime/go-new-channel.c \
runtime/go-new-map.c \
runtime/go-new.c \
- runtime/go-note.c \
runtime/go-panic.c \
runtime/go-print.c \
runtime/go-rec-big.c \
@@ -474,6 +479,7 @@
runtime/go-unsafe-pointer.c \
runtime/go-unwind.c \
runtime/cpuprof.c \
+ $(runtime_lock_files) \
runtime/mcache.c \
runtime/mcentral.c \
$(runtime_mem_file) \
diff -r 8b1402d2f0d3 libgo/runtime/cpuprof.c
--- a/libgo/runtime/cpuprof.c Tue Nov 22 12:16:53 2011 -0800
+++ b/libgo/runtime/cpuprof.c Tue Nov 22 12:18:39 2011 -0800
@@ -115,12 +115,6 @@
static bool evict(Profile*, Entry*);
static bool flushlog(Profile*);
-void
-runtime_cpuprofinit(void)
-{
- runtime_initlock(&lk);
-}
-
// LostProfileData is a no-op function used in profiles
// to mark the number of profiling stack traces that were
// discarded due to slow data writers.
diff -r 8b1402d2f0d3 libgo/runtime/go-main.c
--- a/libgo/runtime/go-main.c Tue Nov 22 12:16:53 2011 -0800
+++ b/libgo/runtime/go-main.c Tue Nov 22 12:18:39 2011 -0800
@@ -47,11 +47,10 @@
g = &runtime_g0;
m->curg = g;
g->m = m;
- runtime_initpanic ();
runtime_mallocinit ();
- runtime_cpuprofinit ();
__go_gc_goroutine_init (&argc);
+ runtime_osinit();
runtime_goargs();
runtime_goenvs();
diff -r 8b1402d2f0d3 libgo/runtime/go-nanotime.c
--- a/libgo/runtime/go-nanotime.c Tue Nov 22 12:16:53 2011 -0800
+++ b/libgo/runtime/go-nanotime.c Tue Nov 22 12:18:39 2011 -0800
@@ -2,7 +2,7 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
-// Return time in nanoseconds. This is only used for computing runtime.
+// Return current time in nanoseconds.
#include
diff -r 8b1402d2f0d3 libgo/runtime/go-note.c
--- a/libgo/runtime/go-note.c Tue Nov 22 12:16:53 2011 -0800
+++ /dev/null Thu Jan 01 00:00:00 1970 +
@@ -1,74 +0,0 @@
-/* go-note.c -- implement notesleep, notewakeup and noteclear.
-
- Copyright 2009 The Go Authors. All rights reserved.
- Use of this source code is governed by a BSD-style
- license that can be found in the LICENSE file. */
-
-/* A note is a one-time notification. noteclear clears the note.
- notesleep waits for a call to notewakeup. notewakeup wakes up
- every thread waiting on the note. */
-
-#include "go-assert.h"
-#include "runtime.h"
-
-/* We use a single global lock and condition variable. It would be
- better to use a futex on GNU/Linux. */
-
-static pthread_mutex_t note_lock = PTHREAD_MUTEX_INITIALIZER;
-static pthread_cond_t note_cond = PTHREAD_COND_INITIALIZER;
-
-/* noteclear is called before any calls to notesleep or
- notewakeup. */
-
-void
-runtime_noteclear (Note* n)
-{
- int32 i;
-
- i = pthread_mutex_lock (¬e_lock);
- __go_assert (i == 0);
-
- n->woken = 0;
-
- i = pthread_mutex_unlock (¬e_lock);
- __go_assert (i == 0);
-}
-
-/* Wait until notewakeup is called. */
-
-void
-runtime_notesleep (Note* n)
-{
- int32 i;
-
- i = pthread_mutex_lock (¬e_lock);
- __go_assert (i == 0);
-
- while (!n->woken)
-{
- i = pthread_cond_wait (¬e_cond, ¬e_lock);
- __go_assert (i == 0);
-}
-
- i = pthread_mutex_unlock (¬e_lock);
- __go_assert (i == 0);
-}
-
-/* Wake up every thread sleeping on the note. */
-
-void
-runtime_notewakeup (Note *n)
-{
- int32 i;
-
- i = pthread_mutex_lock (¬e_lock);
- __go_assert (i == 0);
-
- n->woken = 1;
-
- i = pthread_cond_broadcast (¬e_cond);
- __go_assert (i == 0);
-
- i = pthread_mutex_unlock (¬e_lock);
- __go_assert (i == 0);
-}
diff -r 8b1402d2f0d3 libgo/runtime/go-semacquire.c
--- a/libgo/runtime/go-semacquire.c Tue Nov 22 12:16:53 2011 -0800
+++ b/libgo/runtime/go-semacquire.c Tue Nov 22 12:18:39 2011 -0800
@@ -117,35 +117,3 @@
__go_assert (i == 0);
}
}
-
-
-#ifndef HAVE_SYNC_FETCH_AND_ADD_4
-
-/* For targets which don't have the required sync support. Really
- this should be provided by gcc itself. FIXME. */
-
-static pthread_mutex_t sync_lock = PTHREAD_MUTEX_INITIALIZER;
-
-uint32
-__sync_fetch_and_add_4(uint32*, uint32)
- __attribute__((visibility("hidden")));
-
-u
