Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
On Sun, May 29, 2011 at 03:34:01PM +0200, fredrik danerklint wrote: > Until Bert has applied the patch that I sent him, you need to apply the > following to get masterfunctionallity to work with MongoDB backend: It has been merged (and is available on powerdnssec.org/downloads). Bert ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
> So, please, be so kind as to answer one more question: When PDNS queries
> backends for updated serials, does it provide them with a list of the
> 'notified_serial's (as per your explanation in a previous post of
> yours), that is, a list of all serials which it has previously used in
> notifications
Yes. Everytime PowerDNS asks a backend it will get back a list of domains that
has a different notified_serial than the serial for the domain.
It's the seventh linefrom bottom in the file private.cc that has the "logic"
for including a domain to that list.
Until Bert has applied the patch that I sent him, you need to apply the
following to get masterfunctionallity to work with MongoDB backend:
Index: private.cc
===
--- private.cc (revision 2199)
+++ private.cc (arbetskopia)
@@ -28,6 +28,7 @@
di->id = mongo_r->getIntField("domain_id");
di->last_check = mongo_r->getIntField("last_check");
+ di->notified_serial = mongo_r->getIntField("notified_serial");
if (soadata == NULL)
for( bson::bo::iterator i(mongo_r->getObjectField("masters"));
i.more(); ) {
--
//fredan
___
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
On 27/5/2011 10:16 πμ, Jan-Piet Mens wrote: If it were C, I could help you a bit -- the last time I looked (admittedly a few years ago), PowerDNS was written in C++ ;-) Thank you. Very kind of you. It must be C++. Unfortunately, I am unfamiliar with both (C, C++). I am an old tech guy: FORTRAN, Pascal (back in the late 80's, and in the last years I only have time for some medium internal web apps (mainly procedural and not object-oriented) using: php, coldfusion (some years ago), javascript/html. I also do a bit of basic bash scripting (for our admin tasks). My aim here is to identify problems and (if possible) technical solutions to open issues on the LDAP backend, and request experienced developers to do the implementation. Perhaps I shouldn't, because I am now spending too much time and effort on this, while I have other serious pending tasks, but no one else will, at the moment. Thankfully, Bert has taken over to fix some stuff. I am trying to assist him as much as possible. Thanks again, Nick ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
> I may have to find time to read a bit about programming in C. If it were C, I could help you a bit -- the last time I looked (admittedly a few years ago), PowerDNS was written in C++ ;-) -JP ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
On 27/5/2011 12:59 πμ, fredrik danerklint wrote: Hopefully Bert can answer better than I can on this. Have you read the documentation about writeing a backend? http://doc.powerdns.com/master-backends.html It says that the backend only needs to have these two function to functions as a master backend and that what's I have done with the MongoDB backend. Fredrik, you are right! The documentation states as you say: "Periodically, PDNS queries backends for domains that may have changed, and sends out notifications for slave nameservers." It also says: "Some backends may be able to detect zone changes, others may choose to let the operator indicate which zones have changed and which haven't." (http://doc.powerdns.com/master.html) So, please, be so kind as to answer one more question: When PDNS queries backends for updated serials, does it provide them with a list of the 'notified_serial's (as per your explanation in a previous post of yours), that is, a list of all serials which it has previously used in notifications, or the backends have to maintain such a list (of previously notified serials)? Sorry, I may be becoming tiresome, but I have a hard time running through the source code to find out this detail. I may have to find time to read a bit about programming in C. I have a hard role to play without a regular maintainer on the LDAP backend. Thank you VERY much for all the details you provide and your cooperation, Nick ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
> On 26/5/2011 3:44 μμ, fredrik danerklint wrote: > > Nick, > > > > To answer all of your questions, please take a look at the source file > > 'communicator.cc'. At the end of that file there is a function called > > "void CommunicatorClass::mainloop(void)" that checks slave and master > > every other 'slave-cycle-interval' seconds (parameter taken from the > > pdns.conf file). I assume that this function is running in a seperate > > thread. > > > > This is how PowerDNS knows when to send a update to other nameservers. > > Actually, I am afraid things do not work like that. And I'm afraid that it does work like this. Hopefully Bert can answer better than I can on this. Please take a look at the file that I sad earlier that you should. Have you read the documentation about writeing a backend? http://doc.powerdns.com/master-backends.html It says that the backend only needs to have these two function to functions as a master backend and that what's I have done with the MongoDB backend. -- //fredan ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
On 26/5/2011 3:44 μμ, fredrik danerklint wrote: Nick, To answer all of your questions, please take a look at the source file 'communicator.cc'. At the end of that file there is a function called "void CommunicatorClass::mainloop(void)" that checks slave and master every other 'slave-cycle-interval' seconds (parameter taken from the pdns.conf file). I assume that this function is running in a seperate thread. This is how PowerDNS knows when to send a update to other nameservers. Actually, I am afraid things do not work like that. The 'slave-cycle-interval' parameter is only used by slaves and only when they (the slaves) are in undetermined state, i.e. at launch. "Once a domain has been checked, it will not be checked before its SOA refresh timer has expired." What I was asking is how the *Master* knows that the serial in the SOA of one of its zones has changed. If you have not implemented some solution, Master will never know that a serial (in its own backend!) has been changed, unless you manually let it know. The PowerDNS documentation states: "Some backends may be able to detect zone changes, others may choose to let the operator indicate which zones have changed and which haven’t. Consult the documentation for your backend to see how it processes changes in zones." The usual logical solution is the use of triggers (if your backend supports them). Of course, for the Master it doesn't really matter to know that a serial has changed, because it directly refreshes its data (which is retrieved from the database). [ I don't know what happens with cached data, if they are used - LDAP backend doesn't use cached data; the LDAP server takes care of that. ] But the Master will not be able to Notify slaves... They will wait until their refresh interval (specified in the SOA) expires and only then they will ask the Master if serial has changed. So, in fact you don't have what pdns calls Master operation, unless the backend on the Master provides a mechanism to detect serial changes and send Notify messages to slaves. If it doesn't, you must manually or semi-manually send Notify messages, as I am also now doing with LDAP backend, using a cron job to detect externally (i.e. not within the backend) serial changes and to send, when such changes are detected, Notify messages to slaves. But I guess, when you don't have triggers, you could embed in the backend the above procedure. You could define a time parameter (perhaps changeable in the config) which would cycle domains (zones), automatically, as part of the backend process and detect serial changes. As I have explained earlier (in other threads), this works fine for a moderate number of zones. If the number of zones is high, however, it doesn't scale well. Nick. ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
Nick, To answer all of your questions, please take a look at the source file 'communicator.cc'. At the end of that file there is a function called "void CommunicatorClass::mainloop(void)" that checks slave and master every other 'slave-cycle-interval' seconds (parameter taken from the pdns.conf file). I assume that this function is running in a seperate thread. This is how PowerDNS knows when to send a update to other nameservers. > On 25/5/2011 11:25 μμ, fredrik danerklint wrote: > > PowerDNS ask each backend for domains which has a different > > 'notified_serial' than 'serial' for the domain. > > Thanks Fredrik for the info. If I understand it right, 'notified_serial' > is the one PowerDNS knows as current for a domain, and 'serial' is the > one stored in the same domain (in the backend), which may be updated or > not. > > But how/when is this question from PowerDNS triggered? I mean, how > PowerDNS knows *when* to ask the backend for an updated serial? > > > This function is called getUpdatedMasters(vector* domains) > > and exists in the source file 'master.cc' for MongoDB backend. > > I understand that the above function is part of the master (if I name it > correctly) process (i.e. not the backend). So, how is it in a file of > the backend? > > > What PowerDNS except after a call to this function is a list of all > > domains that has a different serialnumber than notified serialnumber. > > > > Every backend can implement this functionallity differently since > > PowerDNS does not know how the information about a domain is stored. > > That's trivial to retrieve, whatever the backend. > > > The good part that you asked about this is that I (you!) found a bug > > which would not fill out the domains that has a different serial number > > against the real serial number. I've sent Bert a patch to be included in > > the source code to fix this. > > I am glad I helped (even without knowing it) for a fix! > > > Nick -- //fredan ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
On 25/5/2011 11:25 μμ, fredrik danerklint wrote: PowerDNS ask each backend for domains which has a different 'notified_serial' than 'serial' for the domain. Thanks Fredrik for the info. If I understand it right, 'notified_serial' is the one PowerDNS knows as current for a domain, and 'serial' is the one stored in the same domain (in the backend), which may be updated or not. But how/when is this question from PowerDNS triggered? I mean, how PowerDNS knows *when* to ask the backend for an updated serial? This function is called getUpdatedMasters(vector* domains) and exists in the source file 'master.cc' for MongoDB backend. I understand that the above function is part of the master (if I name it correctly) process (i.e. not the backend). So, how is it in a file of the backend? What PowerDNS except after a call to this function is a list of all domains that has a different serialnumber than notified serialnumber. Every backend can implement this functionallity differently since PowerDNS does not know how the information about a domain is stored. That's trivial to retrieve, whatever the backend. The good part that you asked about this is that I (you!) found a bug which would not fill out the domains that has a different serial number against the real serial number. I've sent Bert a patch to be included in the source code to fix this. I am glad I helped (even without knowing it) for a fix! Nick ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
Nikolas, PowerDNS ask each backend for domains which has a different 'notified_serial' than 'serial' for the domain. This function is called getUpdatedMasters(vector* domains) and exists in the source file 'master.cc' for MongoDB backend. What PowerDNS except after a call to this function is a list of all domains that has a different serialnumber than notified serialnumber. Every backend can implement this functionallity differently since PowerDNS does not know how the information about a domain is stored. The good part that you asked about this is that I (you!) found a bug which would not fill out the domains that has a different serial number against the real serial number. I've sent Bert a patch to be included in the source code to fix this. > On 14/4/2011 9:30 μμ, bert hubert wrote: > > Hi Fredrik, > > I have just merged it with the build system based on your latest version. > > It is part of build 2163, and will be shipped as 'experimental' with > > version 3.0. > > Hi Fredrik and all, > > I see here: http://doc.powerdns.com/mongo.html that pdns/mongodb > supports Master functionality. I assume this means that it detects zone > serial changes and sends Notify messages as needed (and then sends AXFR > as requested by slaves). > > My question is: how to you implement this functionality (conceptually), > since Mongo DB (as far as I know) doesn't support triggers? > > Can you provide some feedback on this please? > > Thanks, > Nick -- //fredan ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
Re: [Pdns-users] MongoDB Backend merged Re: Mongo DB and PowerDNS part 3: Now with DNSSEC
On 14/4/2011 9:30 μμ, bert hubert wrote: Hi Fredrik, I have just merged it with the build system based on your latest version. It is part of build 2163, and will be shipped as 'experimental' with version 3.0. Hi Fredrik and all, I see here: http://doc.powerdns.com/mongo.html that pdns/mongodb supports Master functionality. I assume this means that it detects zone serial changes and sends Notify messages as needed (and then sends AXFR as requested by slaves). My question is: how to you implement this functionality (conceptually), since Mongo DB (as far as I know) doesn't support triggers? Can you provide some feedback on this please? Thanks, Nick ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com http://mailman.powerdns.com/mailman/listinfo/pdns-users
