Thank you for the input.
I really don't know what to conclude from this. We have 4 main branches
of tomato: MIPSR1 (very old) MIPSR2 (old) ARM (relatively modern with
medium performance) and especially tomato64 (high performance) that runs
kernel 6.12.85. So even on tomato64 installed in a VM with proper x86 HW
and few GBs of RAM, we still hit the limit which is not related to
system RAM capacity. I have been keeping track of this command here
below, and it seems like we are unable to go past the 64MB mark.
/root# grep 'VmRSS\|VmPeak' /proc/$(ps | grep '[d]nsmasq' | awk
'{print $1}')/status
VmPeak: 57084 kB
VmRSS: 55296 kB
I scanned through the code and even asked support from AI to help me
identify anything that would introduce a hard limit or even a flaw in
the code logic, but nothing was found. BTW you mention dnsmasq v2.90
this has always been our issues; I got involved in the adblock part of
the project about 5 years ago.
So I'm really unsure on why we experience such a limit where you don't?
The block-file solution should still, regardless, be a big improvement.
It's a targeted solution, it improves loading time, resource utilization
of RAM and storage, DNS response time. Finally, it introduces the very
much needed logging of blocked domains only.
On 21/05/2026 21:43, [email protected] wrote:
On Thu, 21 May 2026 19:42:55 +0200
Ercolino de Spiacico <[email protected]> wrote:
In dnsmasq, domain filtering (for example, adblocking) currently relies
on built-in directives such as local and address. That approach works
well up to a point, but it does not scale: around 15 MB (≈700–800k
domains) the process hangs regardless of available system resources.
It could be related to hardware or to the dnsmasq version.
On an x86_64 system running dnsmasq v2.90, I'm currently blocking 1.75M FQDNs
(50MB blacklist* using 'local='; sourced from Univ. of Toulouse). DNsmasq's
virtual size is 152MB; resident size is 147MB. I haven't experienced failures
and have seen only very slightly greater resolution latency. It's been a while
since I last checked, but I think this blacklist increases dnsmasq's memory
requirements by around 150MB.
N
* - dnsmasq.conf (sans comments):
domain-needed
bogus-priv
conf-file=/var/smoothwall/mods/dnsbl/dnsmasq-blacklist.conf
cache-size=1024
To take a different approach I developed this patch in attachment that
adds two new directives: block-file and allow-file. Summary of the
functionality:
- Uses mmap() to load domain lists into RAM.
- Multiple block-file directives are supported.
- Multiple allow-file directives are supported.
- Example syntax:
block-file=/path/to/blockfile#NX,1
NX is optional and can be replaced with any IP address to return instead
of NX.
The trailing ,1 is optional too, and enables logging when domains from
that file are matched (useful to log only blocked domains).
allow-file=/mnt/USB/blockfile-override,1
- Mapped files must be pre-sorted (sorting is not included in this
patch to keep the code minimal).
- Blocking/allowing behaviour mimics dnsmasq’s current semantics. For
example:
dnsmasq config:
local=/example.com/ (blocked)
server=/www.example.com/ (unblocked)
After the patch:
example.com → listed in block-file
www.example.com → listed in allow-file
- Files are loaded one at a time and assesses against the amount of
RAM available. If Current RAM usage + blockfile >= 80% of Total RAM, the
loading is skipped and logged as a warning.
- This code does not change DNS resolution itself; it performs a very
fast name match to decide whether a query should proceed or be answered
with NX or a custom IP.
I developed this patch for FreshTomato router (dnsmasq v2.93rc1). On
those small devices I performed a functional test loading 2.2M domains
(55 MB blockfile) and reload the configuration without interruptions; no
packet drops were observed during a config when reloading. This is
already 3 time the current limit, but I haven't performed any proper
capacity testing yet, which is expected to be much higher.
Any comments or considerations are greatly appreciated.
Thanks.
_______________________________________________
Dnsmasq-discuss mailing list
[email protected]
https://lists.thekelleys.org.uk/cgi-bin/mailman/listinfo/dnsmasq-discuss
