I recently started getting the "sorry, too many clients already" error.
There are currently four places that can generate it, but fortunately
log_error_verbosity was set to verbose so I was able to see that in this
case the warning was generated by proc.c:InitProcess().
But it's still not much, because there are three different lists you can
run out of and get the same error message from InitProcess(). I was
lucky to be able to rule out two of them. max_connections is set to much
more than the sum of possible connections from all relevant pgBouncer
instances we have, so it's not hitting max_connections. Also, this is on
9.4 and we don't use any funny extensions, so it's probably not running
out of bgworkerFreeProcs either. autovacFreeProcs is what's left but
this is still just a guess and I'd prefer to actually know.
I've made a hack for myself (attached diff against 9.4) which adds a
DETAIL-level message telling me which proc list was saturated. It's not
committable in its current form because of a C99 feature and perhaps for
other reasons.
By the way, I've also noticed that the InitProcess() can segfault upon
hitting set_spins_per_delay(procglobal->spins_per_delay). This only
happens when I run REL9_4_STABLE under gdb, set a breakpoint on
InitProcess, see an autovacuum launcher hit the breakpoint and tell gdb
to continue. "p procglobal->spins_per_delay" says "Cannot access memory
at address 0xf01b2f90". Maybe this means nothing.
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index e608198..613b36a 100644
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -279,6 +279,14 @@ InitProcess(void)
{
/* use volatile pointer to prevent code rearrangement */
volatile PROC_HDR *procglobal = ProcGlobal;
+#define LIST_TYPE_ENTRY(e) [(e)] = (#e)
+ enum listType { autovac, bgworker, backend } which;
+ char const * const listTypeNames[] = {
+ LIST_TYPE_ENTRY(autovac),
+ LIST_TYPE_ENTRY(bgworker),
+ LIST_TYPE_ENTRY(backend)
+ };
+#undef LIST_TYPE_ENTRY
/*
* ProcGlobal should be set up already (if we are a backend, we inherit
@@ -309,11 +317,11 @@ InitProcess(void)
set_spins_per_delay(procglobal->spins_per_delay);
if (IsAnyAutoVacuumProcess())
- MyProc = procglobal->autovacFreeProcs;
+ which = autovac, MyProc = procglobal->autovacFreeProcs;
else if (IsBackgroundWorker)
- MyProc = procglobal->bgworkerFreeProcs;
+ which = bgworker, MyProc = procglobal->bgworkerFreeProcs;
else
- MyProc = procglobal->freeProcs;
+ which = backend, MyProc = procglobal->freeProcs;
if (MyProc != NULL)
{
@@ -330,13 +338,13 @@ InitProcess(void)
/*
* If we reach here, all the PGPROCs are in use. This is one of the
* possible places to detect "too many backends", so give the standard
- * error message. XXX do we need to give a different failure message
- * in the autovacuum case?
+ * error message.
*/
SpinLockRelease(ProcStructLock);
ereport(FATAL,
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
- errmsg("sorry, too many clients already")));
+ errmsg("sorry, too many clients already"),
+ errdetail("%s proc list saturated", listTypeNames[which])));
}
MyPgXact = &ProcGlobal->allPgXact[MyProc->pgprocno];
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
