There is currently a memory leak in 2.9.21 from gmysql backend threads dying in this fashion. The threads themselves are not being cleaned up. this is only noticeable on very light loads where the backends do not get enough traffic to keep the mysql connection open. You will notice it by abnormally high virtual memory usage after pdns has been running for a time. Each backend that dies leaves 10MB of virtual memory allocated (the thread stack?) and this will not get cleaned up until pdns is restarted.

Hopefully they will implement mysql connection reconnects in the future, which solves both the memory leak and the backends failing when the connection is closed. The only solution I found was to apply a patch and compile from source.

----- Original Message ----- From: "Kenneth Marshall" <[EMAIL PROTECTED]>
To: "Catalin Constantin" <[EMAIL PROTECTED]>
Cc: <Pdns-users@mailman.powerdns.com>
Sent: Tuesday, January 29, 2008 5:25 AM
Subject: Re: [Pdns-users] Backend error: Failed to execute mysql_query,perhaps connection died? Err=1: MySQL server has gone away


On Tue, Jan 29, 2008 at 03:04:46PM +0200, Catalin Constantin wrote:
Hello,

This is a funny question.
Mysql is FINE and it's up and running on localhost.

Here is the startup log of PDNS.
Jan 29 14:59:05 k2 pdns[30294]: PowerDNS 2.9.20 (C) 2001-2006
PowerDNS.COMBV (Mar 10 2007, 00:36:58, gcc
4.1.2 20061115 (prerelease) (Debian 4.1.1-21)) starting up
Jan 29 14:59:05 k2 pdns[30294]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to
the terms of the GPL version 2.
Jan 29 14:59:05 k2 pdns[30294]: Set effective group id to 109
Jan 29 14:59:05 k2 pdns[30294]: Set effective user id to 113
Jan 29 14:59:05 k2 pdns[30294]: Creating backend connection for TCP
Jan 29 14:59:05 k2 pdns[30294]: Master/slave communicator launching
Jan 29 14:59:05 k2 pdns[30294]: gmysql Connection succesful
Jan 29 14:59:05 k2 pdns[30294]: gmysql Connection succesful
Jan 29 14:59:05 k2 pdns[30294]: About to create 10 backend threads for UDP
Jan 29 14:59:05 k2 pdns[30294]: gmysql Connection succesful
Jan 29 14:59:05 k2 pdns[30294]: 14 slave domains need checking
Jan 29 14:59:05 k2 pdns[30294]: Domain test1.com is stale, master serial
1201611548, our serial 1201610827
Jan 29 14:59:05 k2 pdns[30294]: Domain test2.com is stale, master serial
1201611548, our serial 1201610827
Jan 29 14:59:05 k2 pdns[30294]: Domain test3.com is stale, master serial
1201611548, our serial 1201610827
Jan 29 14:59:05 k2 pdns[30294]: gmysql Connection succesful
Jan 29 14:59:05 k2 pdns[30294]: AXFR started for 'test1.com', transaction
started
Jan 29 14:59:05 k2 pdns[30294]: gmysql Connection succesful
Jan 29 14:59:05 k2 pdns[30294]: AXFR done for 'test1.com', zone committed


The problem is that mysql closes the connection if there is no activity.
PDNS does not make sure that MySQL is still "alive" and from time to time it
shows in the log errors like:
Jan 29 11:33:57 k2 pdns[28941]: Backend error: Failed to execute
mysql_query, perhaps connection died? Err=1: MySQL server has gone away

Does PDNS have a timeout for backend connection "recheck" ?
Can this be configured ?

Is there anything we can do to make sure the connection does not go away ?

Note: the NS server activity for now is not much (we're still testing and
just have a couple of domains on it).

Thanks,

Dear Ms. Constantin,

If a properly functioning application opens a connection to a database,
the connection should not drop until closed. The typical fix is to
increase your timeout on your MySQL server appropriately. This same
thread comes up periodically. In order for PDNS to notice that a
connection has died, it tries to use it with a query. This causes
the errors in your logs. The amount of errors will be proportional
to how long your timeout is set. I certainly would not like to see
a patch that added overhead to every query to check the status of
a connection and re-establish if needed. PDNS will re-connect if
the connection drops, but you will get an error. If your MySQL DB
is not dedicated to PDNS, then you will have to balance the needs
of all of the applications using the DB. It sounds like you may
be running some pretty poorly written ones. Good luck.

Ken

On 1/29/08, Nico van Royen <[EMAIL PROTECTED]> wrote:
> Hello,
>
> Do you infact have a running mysql server ? Also check the database > that
> you have specified in your pdns.conf matches your actual database
> information (db name, user/password etc).
>
> Regards,
> Nico
> WARP3.COM
> ________________________________
> From: Catalin Constantin [mailto:[EMAIL PROTECTED]
> To: pdns-users@mailman.powerdns.com
> Sent: Tue, 29 Jan 2008 10:48:09 +0100
> Subject: [Pdns-users] Backend error: Failed to execute mysql_query,
perhaps
> connection died? Err=1: MySQL server has gone away
>
>
> Hello,
>
> We're getting quite many errors:
> Backend error: Failed to execute mysql_query, perhaps connection died?
> Err=1: MySQL server has gone away latelly.
>
> I checked the docs and the mailing lists for this error and i found
> little informations about it.
>
> What is actually the solution for this error not to happen ?
> Nice would be to have a real example.
>
> Note:
> - we use latest pdns package from Debian Etch (stable).
>
> Thanks,
>
> --
> Catalin Constantin
> Dazoot Software
> http://www.dazoot.eu/
> _______________________________________________
> Pdns-users mailing list
> Pdns-users@mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users
>
> _______________________________________________
> Pdns-users mailing list
> Pdns-users@mailman.powerdns.com
> http://mailman.powerdns.com/mailman/listinfo/pdns-users
>
>


--
Catalin Constantin
Dazoot Software
http://www.dazoot.eu/

_______________________________________________
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
http://mailman.powerdns.com/mailman/listinfo/pdns-users

_______________________________________________
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
http://mailman.powerdns.com/mailman/listinfo/pdns-users


_______________________________________________
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
http://mailman.powerdns.com/mailman/listinfo/pdns-users

Reply via email to