Hey Simon,
On Thu, 2021-04-01 at 23:55 +0100, Simon Kelley wrote:
> I could do with a handle on exactly how people are configuring dnsmasq
> to do ad blocking. It's not something I have much experience of.
The situation for Pi-hole (a popular ad blocker based on dnsmasq) is the
following:
Traditionally, Pi-hole used "addn-hosts" to add HOSTS-like files containing
domains (example:
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts). This
list contains roughly 80,000 domains. This is doable with dnsmasq on all
platforms having at least 512 MB of memory. However, Pi-hole users
typically want to take it to the extremes. They added more and more lists,
often going beyond the one million domains mark. This became a problem
regarding memory. I don't recall complains about slow replies, though.
Anyway, as this became more and more an issue and since we wanted to have
something more professional than a text file (so users can easily add
comments, etc.), we amended the dnsmasq code with an interface to a SQLite3
database holding all domains to be blocked. We also added support for
regular expressions (and hereby wildcards). With this new approach, we
stopped storing anything about blocked domains in dnsmasq's cache: blocked
domains are short-circuited and replied to with a mock answer. They are
never added to the cache. This is done because we allow different lists to
be assigned to different clients so some devices using the DNS server can
be limited further down while other may be fully open on the same process.
This works really fast because the balanced-tree (B-tree) index on the
domain is very efficient. The tree lives transparently in page cache so
accessing it is very fast even in the 1 mio. range (lookup speed scales
logarithmic, typically < 5 ms on Raspberry Pis for 3mio. blocked domains).
Note that we are hooking into dnsmasq's code from "outside" to keep changes
in the dnsmasq codebase minimal so we can straightaway apply any patches
from dnsmasq's git.
So even when this is a bit outside of the current discussion, I thought
it'd be interesting to mention that Pi-hole used to use "addn-hosts" but
stopped to do so some time ago.
Best,
Dominik
___
Dnsmasq-discuss mailing list
Dnsmasq-discuss@lists.thekelleys.org.uk
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
