On Wed, Sep 14, 2011 at 9:12 PM, Lonni J Friedman <netll...@gmail.com> wrote: > On Wed, Sep 14, 2011 at 6:00 PM, Tatsuo Ishii <is...@sraoss.co.jp> wrote: >>> On Wed, Sep 14, 2011 at 4:22 PM, Tatsuo Ishii <is...@sraoss.co.jp> wrote: >>>>>>> I'm pretty sure that's not the case as the messages stop whenever >>>>>>> pgpool isn't running, they were not present prior to using pgpool, and >>>>>>> pg_hba.conf is setup such that the database servers only accept >>>>>>> connections from each other, and the server running pgpool. None of >>>>>>> these servers have normal users connected directly to them (such as >>>>>>> with ssh), nor are they running anything that would connect to the >>>>>>> database as a client. Also, the volume of these messages are such >>>>>>> that something significant has to be causing them. Last night, in the >>>>>>> span of 5 minutes, there were 117 of these messages. >>>>>> >>>>>> Ok. I would like to narraow down the reason why we have "unexpected >>>>>> EOF on client connection" message frequently. I think currently there >>>>>> are two possiblities: >>>>>> >>>>>> 1) pgpool child was killed by some unknown reason(we can omit >>>>>> segfault case because you don't see it in the pgpool log) >>>>>> >>>>>> 2) pgpool child disconnects to PostgreSQL in ungraceful manner >>>>>> >>>>>> For 1) I would like to know if pgpool child process are fine since >>>>>> they are spawned. Are you seeing any pgpool child process disappeared >>>>>> since pgpool started? >>>>> >>>>> I assume this should be determined by num_init_children (which I've >>>>> set to 195 in pgpool.conf)? If so, then I currently have 195 >>>>> processes in either the "wait for connection request" state or >>>>> actively connected state. >>>> >>>> No. Pgpool parent process automatically respawns child process if it's >>>> dyning. So having num_init_children child process is not showing >>>> anything usefull. You record 195 process ids and compare current >>>> process ids. If some of them have been changed, we can assume that >>>> child process is dying. >>> >>> Ah, good point. I just diffed the list of PIDs associated with pgpool >>> processes before and after another EOF message in the log, and there >>> were no differences. So I think that rules out any processes dying? >> >> Right. >> >>> One other thing that I just noticed from comparing logs between all of >>> the database servers is that the time stamps for every one of the >>> 'unexpected EOF on client connection' instances are identical. In >>> other words, they are happening at the same time on each server. I >>> think this further suggests that pgpool has to be doing it? >> >> Yes, I think so unless you set connection_life_time to other than 0 or >> the network connection between PostgreSQL and pgpool is unstable.
I've used both pgpool and pgbouncer. We also have recurring similar client-EOF messages in both pgpool and pgbouncer logs. On pgbouncer, the actual error is: LOG C-0x7db270: mydb/myuser@10.0.0.160:43057 closing because: client unexpected eof (age=44005) On pgpool side, the error is exactly the same as Lonni's. So, from that, I had concluded that it was from our app side. But I have nothing further to substantiate that conclusion. Just adding it as another datapoint. Lonni, given how easy pgbouncer is to setup, it may be worth doing a quick proof-of-concept with it and see if you get similar EOF errors there. If so, I think that would eliminate the problem stemming from the middleware? johnny _______________________________________________ Pgpool-general mailing list Pgpool-general@pgfoundry.org http://pgfoundry.org/mailman/listinfo/pgpool-general