Hi,
I found in slony log files that the worker has restarted the process on
each host in my configuration (1 provider, 3 receivers) at the same time :
2009-04-28 08:47:37 CEST INFO remoteWorkerThread_101: SYNC 4516360
sync_event timing: pqexec (s/count)- provider 0.001/1 - subscriber
0.035/1 - IUD 0.000/2
2009-04-28 08:47:38 CEST ERROR remoteListenThread_102: "select
ev_origin, ev_seqno, ev_timestamp, ev_snapshot,
"pg_catalog".txid_snapshot_xmin(ev_snapshot),
"pg_catalog".txid_snapshot_xmax(ev_snapshot), ev_type,
ev_data1, ev_data2, ev_data3, ev_data4, ev_data5,
ev_data6, ev_data7, ev_data8 from
"_pns_slony_voila_archi_0".sl_event e where (e.ev_origin = '102' and
e.ev_seqno > '29') or (e.ev_origin = '103' and e.ev_seqno > '62') order
by e.ev_origin, e.ev_seqno limit 40" - FATAL: terminating connection
due to administrator command
2009-04-28 08:47:38 CEST ERROR remoteListenThread_102: "select
con_origin, con_received, max(con_seqno) as con_seqno,
max(con_timestamp) as con_timestamp from
"_pns_slony_voila_archi_0".sl_confirm where con_received <> 104 group by
con_origin, con_received" 2009-04-28 08:47:38 CEST CONFIG slon: child
terminated status: 11; pid: 28152, current worker pid: 28152
2009-04-28 08:47:38 CEST CONFIG slon: restart of worker in 10 seconds
The origin of the first error is that postgresql has been restarted on
one receiver without stopping slon before. Is it a known source of errors ?
The slon process has been terminated with status=0 on the receiver where
postgresql has been restarted and has segfault (chid status=11) on the
others (each watchdog has restart a new slon process). This fact seems
to cause the error with events and confirmations.
Regards.
Cyril Scetbon wrote:
A simple restart of all slon processes seems to have resolved the
issue. weird ....
Cyril Scetbon wrote:
I use test_slony_state and see some informations like :
Check of event info
---------------------------------------------------
Problem : Events not propagating to node 2
Problem : Events not propagating to node 4
Problem : Events not propagating to node 3
Check of sl_confirm aging
---------------------------------------------------
Confirmations not propagating from 2 to 1
Confirmations not propagating from 2 to 3
Confirmations not propagating from 2 to 4
Confirmations not propagating from 3 to 1
Confirmations not propagating from 3 to 2
Confirmations not propagating from 3 to 4
Confirmations not propagating from 4 to 1
Confirmations not propagating from 4 to 2
Confirmations not propagating from 4 to 3
You can see the results on one of my databases :
- for Confirmations
select con_origin, con_received, min(con_seqno) as minseq,
max(con_seqno) as maxseq, date_trunc('minutes',
min(now()-con_timestamp)) as age1, date_trunc('minutes',
max(now()-con_timestamp)) as age2, min(now() - con_timestamp) >
'00:30:00' as tooold from _pns_slony_voila_archi_0.sl_confirm group
by con_origin, con_received order by con_origin, con_received;
con_origin | con_received | minseq | maxseq | age1 | age2 |
tooold
------------+--------------+---------+---------+----------+----------+--------
101 | 102 | 4464029 | 4464792 | 00:00:00 | 00:16:00 | f
101 | 103 | 4464027 | 4464792 | 00:00:00 | 00:16:00 | f
101 | 104 | 4464024 | 4464792 | 00:00:00 | 00:16:00 | f
102 | 101 | 29 | 29 | 03:39:00 | 03:39:00 | t
102 | 103 | 29 | 29 | 03:39:00 | 03:39:00 | t
102 | 104 | 29 | 29 | 03:39:00 | 03:39:00 | t
103 | 101 | 62 | 62 | 03:39:00 | 03:39:00 | t
103 | 102 | 62 | 62 | 03:39:00 | 03:39:00 | t
103 | 104 | 62 | 62 | 03:39:00 | 03:39:00 | t
104 | 101 | 57 | 57 | 03:39:00 | 03:39:00 | t
104 | 102 | 57 | 57 | 03:39:00 | 03:39:00 | t
104 | 103 | 57 | 57 | 03:39:00 | 03:39:00 | t
- for Events
select ev_origin, min(ev_seqno), max(ev_seqno),
date_trunc('minutes', min(now() - ev_timestamp)),
date_trunc('minutes', max(now() - ev_timestamp)),
min(now() - ev_timestamp) > '00:30:00' as agehi
from _pns_slony_voila_archi_0.sl_event group by ev_origin;
ev_origin | min | max | date_trunc | date_trunc | agehi
-----------+---------+---------+------------+------------+-------
103 | 62 | 62 | 03:49:00 | 03:49:00 | t
104 | 57 | 57 | 03:49:00 | 03:49:00 | t
102 | 29 | 29 | 03:49:00 | 03:49:00 | t
101 | 4464493 | 4465346 | 00:00:00 | 00:14:00 | f
What can be the source of these errors and how to track them ? FYI, I
have logs but in debug level 1.
I did not have issues in 1.2.15.
Regards.
--
Cyril SCETBON
_______________________________________________
Slony1-general mailing list
[email protected]
http://lists.slony.info/mailman/listinfo/slony1-general
