Re: Logging Response Results

[Mats Dufberg]

The .SE Registry has created a solution that stores queries and answers.



PacketQ (replaces DNS2DB)

PacketQ is a tool for analyzing PCAP-data, if can work with any packets 
but is designed primarily for DNS and ICMP-traffic. PacketQ reads, 
filters and groups the packets read from the PCAP-files using standard 
SQL-queries. The tool is built in C for performance and portability. The 
distribution also includes a simple interactive GUI for analyzing the 
collected data.


http://github.com/dotse/packetq




Stefan Certic skrev 2011-06-23 22:27:

Thanks Chuck

Yes, that would be a solution, but i need logs processed through syslog and
stored into database (matching the initial query from query log).

Pharsing tcpdump is not going to be suitable for highly loaded system. I was
more looking for a solution to log responses same way queryes are logged.

Regards,

On Thursday, June 23, 2011 09:44:46 pm Chuck Swiger wrote:

On Jun 23, 2011, at 12:16 PM, Stefan Certic wrote:

Does anyone have idea on following... Apart from bind9 query log, is it
possible to log response returned to client?


Sure: use tcpdump, wireshark, or another network sniffer of your choice and
observe DNS responses to the clients you're interested in.  (Whether this
is better than using query logging is another question entirely.)

Regards,




___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Stephane Bortzmeyer]

On Thu, Jun 23, 2011 at 02:31:22PM -0700,
 Ray Van Dolson  wrote 
 a message of 37 lines which said:

> If you're handy with Python, pcapy[1]

Quite limited.

> and impacket[2] 

No IPv6 support. And, anyway, neither pcapy nor impacket parses the
DNS (if you read French, see 
).

> would likely be a more efficient way to parse DNS traffic for query
> responses than working with tcpdump output natively (unless you're
> skilled with C).

It exists several DNS parsers written in C in free software (I
mentionbed one before but there is also dns2db, the one in dnscap, and
of course the ones in tcpdump and wireshark, etc) so there is no need
to write a C parser from scratch.

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Stephane Bortzmeyer]

On Thu, Jun 23, 2011 at 10:27:31PM +0200,
 Stefan Certic  wrote 
 a message of 65 lines which said:

> stored into database (matching the initial query from query log).

This may help: 

> We monitor our email system and may record your emails.

Don't!
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Stefan Certic]

Unfortunately not, since billing is per query based, and each zone can have 
different pricing. Also, results per query are very important for analytical 
purposes in order to be able to spot problems in case some of forward zones 
stop wroking and/or provide unacceptable sucess rates.

Anyway, i am goiing to try to patch the code to get the results, since query 
log work perfectly for the first part of process.

Thanks for your help.

On Friday, June 24, 2011 12:16:09 am Chuck Swiger wrote:
> On Jun 23, 2011, at 2:28 PM, Stefan Certic wrote:
> > It is Enum server, and logging is taking care of billing process.
> 
> I don't see why you need to preserve queries and responses, unless you plan
> to charge differently for different DNS requests.  Can't you just track
> traffic per client using netflow records, firewall counters, etc?
> 
> Also, it's hard to beat free-- and there are plenty of freely available DNS
> servers around.
> 
> Regards,

-- 
Stefan Certic

RoutoMessaging
48 Charlotte Street
London, W1T 2NS
United Kingdom
http://www.routomessaging.com
GSMA Associate Member

Switchboard +44 (0) 870 231  
Fax + 44 (0) 870 231 7775

Email  : ste...@routotelecom.com
MSN ID : ste...@routotelecom.com
 
DISCLAIMER

This email contains information provided by Routo Telecommunications
Ltd, which may be privileged or confidential. It is meant only for the
individual(s) or entity named above. If you are not the intended
recipient, note that disclosing, copying, distributing or using this
information is prohibited. If you have received this email in error,
please let me know immediately on the email address above.

Routo Telecommunications Ltd may not be held responsible for the
content of this email as it may reflect the personal view of the
sender and not that of the company.

Internet communications cannot be guaranteed to be timely, secure,
error or virus-free. The sender does not accept liability for any
errors or omissions.

We monitor our email system and may record your emails.

Routo Telecommunications Ltd Registration Number 04546322 has its
principal place of business at 48 Charlotte Street, London, W1T 2NS,
United Kingdom.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Chuck Swiger]

On Jun 23, 2011, at 2:28 PM, Stefan Certic wrote:
> It is Enum server, and logging is taking care of billing process.

I don't see why you need to preserve queries and responses, unless you plan to 
charge differently for different DNS requests.  Can't you just track traffic 
per client using netflow records, firewall counters, etc?

Also, it's hard to beat free-- and there are plenty of freely available DNS 
servers around.

Regards,
-- 
-Chuck

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Ray Van Dolson]

On Thu, Jun 23, 2011 at 01:58:37PM -0700, Phil Mayers wrote:
> On 06/23/2011 09:27 PM, Stefan Certic wrote:
> > Thanks Chuck
> >
> > Yes, that would be a solution, but i need logs processed through syslog and
> > stored into database (matching the initial query from query log).
> >
> > Pharsing tcpdump is not going to be suitable for highly loaded system. I was
> > more looking for a solution to log responses same way queryes are logged.
> 
> The problem is that queries and responses are not the same type of 
> thing. A query contains a single question, and is usually relatively 
> small. A response can contain multiple answers, and multiple types of 
> answer, and with DNSSEC they can get big.
> 
> There's no inherent reason parsing tcpdump needs to be slow. It's 
> written in C.
> 
> Anyway: bind itself cannot log answers. You will need to patch the 
> source if you want this.

Don't mean to venture into off-topic territory, but

If you're handy with Python, pcapy[1] and impacket[2] would likely be a
more efficient way to parse DNS traffic for query  responses than
working with tcpdump output natively (unless you're skilled with C).

Ray

[1] http://oss.coresecurity.com/projects/pcapy.html
[2] http://oss.coresecurity.com/projects/impacket.html
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Stefan Certic]

It is Enum server, and logging is taking care of billing process.

Flow is going something like this:

- Accept Query
- Write QueryLog through Syslog
- Syslog do an insert into database.
- Respond to query by asking forwarder or through local master zone
(Everything fine till now)
- Log response sent to client through syslog ???
- Match initial query and update results (will deppend on previous step).

Hope you get the clear picture now.

Thanks,

On Thursday, June 23, 2011 10:42:19 pm Chuck Swiger wrote:
> On Jun 23, 2011, at 1:27 PM, Stefan Certic wrote:
> > Thanks Chuck
> > 
> > Yes, that would be a solution, but i need logs processed through syslog
> > and stored into database (matching the initial query from query log).
> 
> Why do you need to send this information via syslog to a database?
> 
> > Pharsing tcpdump is not going to be suitable for highly loaded system. I
> > was more looking for a solution to log responses same way queryes are
> > logged.
> 
> Parsing tcpdump doesn't constitute much work; I've got scripts which deal
> with NTP traffic at 500 - 2000+ requests per second without consuming much
> resources...although monitoring NTP takes noticeably more work than ntpd
> itself needs to provide time.
> 
> It's the other requirements being added which strike me as heavy-weight.
> 
> Regards,

-- 
Stefan Certic

RoutoMessaging
48 Charlotte Street
London, W1T 2NS
United Kingdom
http://www.routomessaging.com
GSMA Associate Member

Switchboard +44 (0) 870 231  
Fax + 44 (0) 870 231 7775

Email  : ste...@routotelecom.com
MSN ID : ste...@routotelecom.com
 
DISCLAIMER

This email contains information provided by Routo Telecommunications
Ltd, which may be privileged or confidential. It is meant only for the
individual(s) or entity named above. If you are not the intended
recipient, note that disclosing, copying, distributing or using this
information is prohibited. If you have received this email in error,
please let me know immediately on the email address above.

Routo Telecommunications Ltd may not be held responsible for the
content of this email as it may reflect the personal view of the
sender and not that of the company.

Internet communications cannot be guaranteed to be timely, secure,
error or virus-free. The sender does not accept liability for any
errors or omissions.

We monitor our email system and may record your emails.

Routo Telecommunications Ltd Registration Number 04546322 has its
principal place of business at 48 Charlotte Street, London, W1T 2NS,
United Kingdom.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Kevin Darcy]

On 6/23/2011 4:27 PM, Stefan Certic wrote:

Thanks Chuck

Yes, that would be a solution, but i need logs processed through syslog and
stored into database (matching the initial query from query log).

Pharsing tcpdump is not going to be suitable for highly loaded system. I was
more looking for a solution to log responses same way queryes are logged.

Regards,

On Thursday, June 23, 2011 09:44:46 pm Chuck Swiger wrote:

On Jun 23, 2011, at 12:16 PM, Stefan Certic wrote:

Does anyone have idea on following... Apart from bind9 query log, is it
possible to log response returned to client?

Sure: use tcpdump, wireshark, or another network sniffer of your choice and
observe DNS responses to the clients you're interested in.  (Whether this
is better than using query logging is another question entirely.)


The parsing can be done off-line.

Depending on your LAN structure, you might be able to capture the 
packets off-box as well.



- Kevin


___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Phil Mayers]

On 06/23/2011 09:27 PM, Stefan Certic wrote:

Thanks Chuck

Yes, that would be a solution, but i need logs processed through syslog and
stored into database (matching the initial query from query log).

Pharsing tcpdump is not going to be suitable for highly loaded system. I was
more looking for a solution to log responses same way queryes are logged.


The problem is that queries and responses are not the same type of 
thing. A query contains a single question, and is usually relatively 
small. A response can contain multiple answers, and multiple types of 
answer, and with DNSSEC they can get big.


There's no inherent reason parsing tcpdump needs to be slow. It's 
written in C.


Anyway: bind itself cannot log answers. You will need to patch the 
source if you want this.

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Chuck Swiger]

On Jun 23, 2011, at 1:27 PM, Stefan Certic wrote:
> Thanks Chuck
> 
> Yes, that would be a solution, but i need logs processed through syslog and 
> stored into database (matching the initial query from query log).

Why do you need to send this information via syslog to a database?

> Pharsing tcpdump is not going to be suitable for highly loaded system. I was 
> more looking for a solution to log responses same way queryes are logged.

Parsing tcpdump doesn't constitute much work; I've got scripts which deal with 
NTP traffic at 500 - 2000+ requests per second without consuming much 
resources...although monitoring NTP takes noticeably more work than ntpd itself 
needs to provide time.

It's the other requirements being added which strike me as heavy-weight.

Regards,
-- 
-Chuck

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Stefan Certic]

Thanks Chuck

Yes, that would be a solution, but i need logs processed through syslog and 
stored into database (matching the initial query from query log).

Pharsing tcpdump is not going to be suitable for highly loaded system. I was 
more looking for a solution to log responses same way queryes are logged.

Regards,

On Thursday, June 23, 2011 09:44:46 pm Chuck Swiger wrote:
> On Jun 23, 2011, at 12:16 PM, Stefan Certic wrote:
> > Does anyone have idea on following... Apart from bind9 query log, is it
> > possible to log response returned to client?
> 
> Sure: use tcpdump, wireshark, or another network sniffer of your choice and
> observe DNS responses to the clients you're interested in.  (Whether this
> is better than using query logging is another question entirely.)
> 
> Regards,

-- 
Stefan Certic

RoutoMessaging
48 Charlotte Street
London, W1T 2NS
United Kingdom
http://www.routomessaging.com
GSMA Associate Member

Switchboard +44 (0) 870 231  
Fax + 44 (0) 870 231 7775

Email  : ste...@routotelecom.com
MSN ID : ste...@routotelecom.com
 
DISCLAIMER

This email contains information provided by Routo Telecommunications
Ltd, which may be privileged or confidential. It is meant only for the
individual(s) or entity named above. If you are not the intended
recipient, note that disclosing, copying, distributing or using this
information is prohibited. If you have received this email in error,
please let me know immediately on the email address above.

Routo Telecommunications Ltd may not be held responsible for the
content of this email as it may reflect the personal view of the
sender and not that of the company.

Internet communications cannot be guaranteed to be timely, secure,
error or virus-free. The sender does not accept liability for any
errors or omissions.

We monitor our email system and may record your emails.

Routo Telecommunications Ltd Registration Number 04546322 has its
principal place of business at 48 Charlotte Street, London, W1T 2NS,
United Kingdom.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Re: Logging Response Results

[Chuck Swiger]

On Jun 23, 2011, at 12:16 PM, Stefan Certic wrote:
> Does anyone have idea on following... Apart from bind9 query log, is it 
> possible to log response returned to client?

Sure: use tcpdump, wireshark, or another network sniffer of your choice and 
observe DNS responses to the clients you're interested in.  (Whether this is 
better than using query logging is another question entirely.)

Regards,
-- 
-Chuck

___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users

Logging Response Results

[Stefan Certic]

Hi,

Does anyone have idea on following... Apart from bind9 query log, is it 
possible to log response returned to client?

Regards,
-- 
Stefan Certic

RoutoMessaging
48 Charlotte Street
London, W1T 2NS
United Kingdom
http://www.routomessaging.com
GSMA Associate Member

Switchboard +44 (0) 870 231  
Fax + 44 (0) 870 231 7775

Email  : ste...@routotelecom.com
MSN ID : ste...@routotelecom.com
 
DISCLAIMER

This email contains information provided by Routo Telecommunications
Ltd, which may be privileged or confidential. It is meant only for the
individual(s) or entity named above. If you are not the intended
recipient, note that disclosing, copying, distributing or using this
information is prohibited. If you have received this email in error,
please let me know immediately on the email address above.

Routo Telecommunications Ltd may not be held responsible for the
content of this email as it may reflect the personal view of the
sender and not that of the company.

Internet communications cannot be guaranteed to be timely, secure,
error or virus-free. The sender does not accept liability for any
errors or omissions.

We monitor our email system and may record your emails.

Routo Telecommunications Ltd Registration Number 04546322 has its
principal place of business at 48 Charlotte Street, London, W1T 2NS,
United Kingdom.
___
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe 
from this list

bind-users mailing list
bind-users@lists.isc.org
https://lists.isc.org/mailman/listinfo/bind-users