Hi!
Looks like I forgot to attach the logfile ... sorry - so once again:
I have the following issue with PGPool-II:
I tried 3.0.4 and 3.1.0-alpha2 with the same result.
It sometimes happens, that after stopping PGPool, some SHM-segements don't get
freed.
Precisely I'm talking about 5 SHM-segments and 1 semaphore (as reported by
ipcs).
When I then restart pgpool-II, it fails to start with the log-output: "Address
already in use".
This is just the consecutive fault, but the most noticeable.
The real failure seems to happen on shutdown, and - as further investigations
have shown - probably only if a client is still connected to pgpool.
In that case the shm-resources are not released again.
It is possible to trace this whith ipcs. The number of used shm-segments
increase after starting pgpool, and normally revert to the old value after
stopping it again. But if a client is still busy on this database, the number
of used shms doesn't decrease after a shutdown, and the next restart of pgpool
fails.
I have attached this part of the logfile, because it would be too long to have
it inline here.
But an indication might be this line:
2011-06-28 13:14:36 DEBUG: pid 13370: child receives smart shutdown request but
it's not in idle state
(for the rest please have a look in the attachment).
Well, the number of 6 leaks on a restart doesn't seem to be much, but our
system shall run in a high-available environment and it might be necessary that
pgpool needs to be restarted by the cluster software after any process fails.
Also could it be necessary to restart pgpool because of a configuration-change
of the backends.
And during tests we already came to the point, when after 100 restarts it was
not possible to start it any more because no resources where available.
Example of startup-log after 100 restarts: "could not create semaphores".
2011-06-28 15:02:43 DEBUG: pid 6000: num_backends: 2 total_weight: 1.000000
2011-06-28 15:02:43 DEBUG: pid 6000: backend 0 weight: 2147483647.000000
2011-06-28 15:02:43 DEBUG: pid 6000: backend 1 weight: 0.000000
pid file found but it seems bogus. Trying to start pgpool anyway...
2011-06-28 15:02:43 ERROR: pid 6000: pool_init_pool_passwd: couldn't open
/etc/pgpool-II-90/pool_passwd. reason: Permission denied
2011-06-28 15:02:43 ERROR: pid 6000: could not create 3 semaphores: No space
left on device
2011-06-28 15:02:43 ERROR: pid 6000: Unable to create semaphores. Exiting...
2011-06-28 15:02:43 DEBUG: pid 6000: shmem_exit(1)
I have attached a logfile which contains a startup, a shutdown and the
afterwards failing restart.
Maybe anyone can identify the source of this problem.
And BTW: I cannot file this as a bug (or browse the buglist) at
http://pgfoundry.org/tracker/?group_id=1000055
it says: Database Error: ERROR: could not open relation "artifact_message": No
such file or directory
best regards,
Harald
---> STARTUP OF PGPOOL <---
2011-06-28 13:14:32 DEBUG: pid 13366: num_backends: 2 total_weight: 1.000000
2011-06-28 13:14:32 DEBUG: pid 13366: backend 0 weight: 2147483647.000000
2011-06-28 13:14:32 DEBUG: pid 13366: backend 1 weight: 0.000000
2011-06-28 13:14:32 ERROR: pid 13366: pool_init_pool_passwd: couldn't open /etc/pgpool-II-90/pool_passwd. reason: Permission denied
2011-06-28 13:14:32 DEBUG: pid 13368: I am 13368
2011-06-28 13:14:32 DEBUG: pid 13369: I am 13369
2011-06-28 13:14:32 DEBUG: pid 13370: I am 13370
2011-06-28 13:14:32 DEBUG: pid 13371: I am 13371
2011-06-28 13:14:32 DEBUG: pid 13372: I am 13372
2011-06-28 13:14:32 DEBUG: pid 13373: I am 13373
2011-06-28 13:14:32 DEBUG: pid 13374: I am 13374
2011-06-28 13:14:32 DEBUG: pid 13375: I am 13375
2011-06-28 13:14:32 DEBUG: pid 13376: I am 13376
2011-06-28 13:14:32 DEBUG: pid 13377: I am 13377
2011-06-28 13:14:32 DEBUG: pid 13378: I am 13378
2011-06-28 13:14:32 DEBUG: pid 13379: I am 13379
2011-06-28 13:14:32 DEBUG: pid 13380: I am 13380
2011-06-28 13:14:32 DEBUG: pid 13381: I am 13381
2011-06-28 13:14:32 DEBUG: pid 13382: I am 13382
2011-06-28 13:14:32 DEBUG: pid 13383: I am 13383
2011-06-28 13:14:32 DEBUG: pid 13384: I am 13384
2011-06-28 13:14:32 DEBUG: pid 13385: I am 13385
2011-06-28 13:14:32 DEBUG: pid 13386: I am 13386
2011-06-28 13:14:32 DEBUG: pid 13387: I am 13387
2011-06-28 13:14:32 DEBUG: pid 13388: I am 13388
2011-06-28 13:14:32 DEBUG: pid 13389: I am 13389
2011-06-28 13:14:32 DEBUG: pid 13390: I am 13390
2011-06-28 13:14:32 DEBUG: pid 13391: I am 13391
2011-06-28 13:14:32 DEBUG: pid 13392: I am 13392
2011-06-28 13:14:32 DEBUG: pid 13393: I am 13393
2011-06-28 13:14:32 DEBUG: pid 13394: I am 13394
2011-06-28 13:14:32 DEBUG: pid 13395: I am 13395
2011-06-28 13:14:32 DEBUG: pid 13396: I am 13396
2011-06-28 13:14:32 DEBUG: pid 13397: I am 13397
2011-06-28 13:14:32 DEBUG: pid 13398: I am 13398
2011-06-28 13:14:32 DEBUG: pid 13399: I am 13399
2011-06-28 13:14:32 LOG: pid 13366: pgpool-II successfully started. version 3.0.4 (umiyameboshi)
2011-06-28 13:14:32 DEBUG: pid 13401: I am 13401
2011-06-28 13:14:32 DEBUG: pid 13366: find_primary_node: not in streaming replication mode
2011-06-28 13:14:35 DEBUG: pid 13370: I am 13370 accept fd 5
2011-06-28 13:14:35 LOG: pid 13370: connection received: host=10.0.0.66 port=45856
2011-06-28 13:14:35 DEBUG: pid 13370: Protocol Major: 1234 Minor: 5679 database: user:
2011-06-28 13:14:35 DEBUG: pid 13370: SSLRequest from client
2011-06-28 13:14:35 DEBUG: pid 13370: pool_ssl: SSL requested but SSL support is not available
2011-06-28 13:14:35 DEBUG: pid 13370: Protocol Major: 3 Minor: 0 database: hansi user: hansi
2011-06-28 13:14:35 DEBUG: pid 13370: new_connection: connecting 0 backend
2011-06-28 13:14:35 DEBUG: pid 13370: new_connection: connecting 1 backend
2011-06-28 13:14:35 DEBUG: pid 13370: pool_ssl: SSL requested but SSL support is not available
2011-06-28 13:14:35 DEBUG: pid 13370: pool_ssl: SSL requested but SSL support is not available
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 8
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 1 length: 8
2011-06-28 13:14:35 DEBUG: pid 13370: pool_do_auth: auth kind:0
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 22
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 22
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: application_name value:
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: application_name value:
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: client_encoding value: UTF8
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: client_encoding value: UTF8
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 23
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 23
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: DateStyle value: ISO, DMY
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: DateStyle value: ISO, DMY
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: integer_datetimes value: on
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: integer_datetimes value: on
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 27
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 27
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: IntervalStyle value: postgres
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: IntervalStyle value: postgres
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 20
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 20
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: is_superuser value: on
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: is_superuser value: on
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: server_encoding value: UTF8
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: server_encoding value: UTF8
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 25
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: server_version value: 9.0.3
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: server_version value: 9.0.3
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 31
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 31
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: session_authorization value: hansi
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: session_authorization value: hansi
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 36
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 36
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: standard_conforming_strings value: off
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: standard_conforming_strings value: off
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 0 length: 27
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length2: master slot: 1 length: 27
2011-06-28 13:14:35 DEBUG: pid 13370: 0 th backend: name: TimeZone value: Europe/Berlin
2011-06-28 13:14:35 DEBUG: pid 13370: 1 th backend: name: TimeZone value: Europe/Berlin
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 12
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 1 length: 12
2011-06-28 13:14:35 DEBUG: pid 13370: pool_do_auth: cp->info[i]:0x2b50d70d1000 pid:13408
2011-06-28 13:14:35 DEBUG: pid 13370: pool_do_auth: cp->info[i]:0x2b50d70d1088 pid:13409
2011-06-28 13:14:35 DEBUG: pid 13370: pool_send_auth_ok: send pid 13409 to frontend
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_skip_reading_from_backends: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_ignore_till_sync: done
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 1 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 1 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend Q(51)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: statement2: begin transaction
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 0 query: begin transaction
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend C NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: C
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend Q(51)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
---> SHUTDOWN PGPOOL <---
2011-06-28 13:14:35 DEBUG: pid 13370: statement2: DELETE FROM parameter WHERE location = 1 AND action = 1 AND id =1
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 0 query: DELETE FROM parameter WHERE location = 1 AND action = 1 AND id =1
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend C NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: C
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend Q(51)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: statement2: deallocate "prepstmt"
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 0 query: deallocate "prepstmt"
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend C NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: C
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend P(50)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: Parse: statement name <prepstmt>
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 2 query: SELECT location, action, id, progress FROM parameter
2011-06-28 13:14:35 DEBUG: pid 13370: Parse: waiting for master completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 LOG: pid 13370: pool_send_and_wait: Error or notice message from backend: : DB node id: 0 backend pid: 13408 statement: prepstmt message: parse prepstmt: SELECT location, action, id, progress FROM parameter
2011-06-28 13:14:35 DEBUG: pid 13370: detect_error: kind: N
2011-06-28 13:14:35 LOG: pid 13370: Parse: Error or notice message from backend: : DB node id: 0 backend pid: 13408 statement: SELECT location, action, id, progress FROM parameter message: parse prepstmt: SELECT location, action, id, progress FROM parameter
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend N NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: N
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend 1 NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: 1
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend S(53)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend D(44)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 2 query: SELECT location, action, id, progress FROM parameter
2011-06-28 13:14:35 DEBUG: pid 13370: Describe: waiting for master completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend t NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: t
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend T NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: T
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend S(53)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend Q(51)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: statement2: INSERT INTO parameter(location, action, id, progress) VALUES('1','1','1','2')
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 0 query: INSERT INTO parameter(location, action, id, progress) VALUES('1','1','1','2')
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend C NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: C
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessFrontendResponse: kind from frontend Q(51)
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_doing_extended_query_message: done
2011-06-28 13:14:35 DEBUG: pid 13370: statement2: COMMIT WORK
2011-06-28 13:14:35 DEBUG: pid 13370: pool_set_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: send_to_where: 0 query: COMMIT WORK
2011-06-28 13:14:35 DEBUG: pid 13370: wait_for_query_response: waiting for backend 0 completing the query
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend C NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: C
2011-06-28 13:14:35 DEBUG: pid 13370: read_kind_from_backend: read kind from 0 th backend Z NUM_BACKENDS: 2
2011-06-28 13:14:35 DEBUG: pid 13370: ProcessBackendResponse: kind from backend: Z
2011-06-28 13:14:35 DEBUG: pid 13370: pool_read_message_length: slot: 0 length: 5
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:35 DEBUG: pid 13370: pool_unset_query_in_progress: done
2011-06-28 13:14:36 DEBUG: pid 13382: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13378: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13398: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13379: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13381: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13378 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: child 13382 exits with status 0
2011-06-28 13:14:36 LOG: pid 13366: PCP child 13400 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13397: child received shutdown request signal 15
2011-06-28 13:14:36 LOG: pid 13366: fork a new PCP child pid 13422
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13398 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: child 13379 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13380: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: child 13381 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13376: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13399: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13396: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13397 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13377: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13376 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13375: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: child 13380 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13395: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13396 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: child 13399 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13377 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13374: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: child 13375 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13383: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 LOG: pid 13366: worker child 13401 exits with status 0
2011-06-28 13:14:36 LOG: pid 13366: fork a new worker child pid 13401
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13394: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13395 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13373: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13374 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13384: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13383 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13393: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13394 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13372: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13373 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13385: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13384 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13371: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13372 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13392: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: child 13393 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13387: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: child 13385 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13370: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13370: child receives smart shutdown request but it's not in idle state
2011-06-28 13:14:36 DEBUG: pid 13366: child 13371 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13368: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13369: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler called
2011-06-28 13:14:36 DEBUG: pid 13391: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: call wait3
2011-06-28 13:14:36 DEBUG: pid 13366: child 13392 exits with status 0
2011-06-28 13:14:36 DEBUG: pid 13366: reap_handler: normally exited
2011-06-28 13:14:36 DEBUG: pid 13386: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13390: child received shutdown request signal 15
2011-06-28 13:14:36 LOG: pid 13366: received smart shutdown request
2011-06-28 13:14:36 DEBUG: pid 13370: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13390: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13370: child receives smart shutdown request but it's not in idle state
2011-06-28 13:14:36 DEBUG: pid 13389: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13387: child received shutdown request signal 15
2011-06-28 13:14:36 DEBUG: pid 13388: child received shutdown request signal 15
---> RESTART PGPOOL <---
2011-06-28 13:14:40 DEBUG: pid 13480: num_backends: 2 total_weight: 1.000000
2011-06-28 13:14:40 DEBUG: pid 13480: backend 0 weight: 2147483647.000000
2011-06-28 13:14:40 DEBUG: pid 13480: backend 1 weight: 0.000000
pid file found but it seems bogus. Trying to start pgpool anyway...
2011-06-28 13:14:40 ERROR: pid 13480: pool_init_pool_passwd: couldn't open /etc/pgpool-II-90/pool_passwd. reason: Permission denied
2011-06-28 13:14:40 ERROR: pid 13480: bind(/tmp/.s.PGSQL.9999) failed. reason: Address already in use
2011-06-28 13:14:40 ERROR: pid 13480: unlink() failed: No such file or directory
2011-06-28 13:14:40 DEBUG: pid 13480: shmem_exit(1)
_______________________________________________
Pgpool-hackers mailing list
[email protected]
http://pgfoundry.org/mailman/listinfo/pgpool-hackers
